Making audit logs sexy - best practices for audit logging with examples

It might be a sign of my ever-growing age, but I've fallen deeply in love with a robust, detailed audit logging system. For all of my future projects, it'll be one of the first features I build. Knowing who did what, when they did it, and what was changed, is hugely powerful in not only debugging business logic, but getting a quick glance at a user's account activity and understanding how they're using your software.

When searching for "best practices for audit logging", I didn't find anything, so I thought I'd try fill that void with my experience in how massively helpful a good audit log is, and how to go about building it.

What are audit logs?

Audit logs are effectively an activity log of things that happened in a given entity (usually an "account").

For instance, for one of my side projects - reservadesk.com, we log all activity within an "organisation" so that admin members (and us internally) who manage the organisation can see who changed what, and when. It makes it very easy to debug customer problems, as well as simply being able to track active accounts and see what people are doing as they poke about at your app.

What's more, a lot of large clients will expect this kind of thing, and are willing and able to pay for it.

What audit logs are NOT

Audit logs are not like console logs in the console, or backend logging in your API like you might have in Datadog, Rapid7 or some other log provider. Those type of logs have equal, if not more, importance. But they serve a different purpose. Those types of logs are for your developers or technical support to interrogate and check for problems. Audit logging is for your customers themselves (so they can see changes to things over time) and also your support/success staff for issue resolution and, to a lesser degree, usage.

Learning from Flipdish

Flipdish is my day job. I work in Product and at the time of writing, we just reached Unicorn status with our latest round of investment. Since joining three years ago, it's been a whirlwind, to say the least, so right now feels like a good opportunity to pause and reflect on some of the things I've learned.

Here's that list. One of the bigger things on that list audit logs. When a customer calls in with a query, being able to track down who changed what, and when, is a powerful tool to have in your arsenal to get any issues resolved as quickly and painlessly as possible.

BTW - I'm hiring for product managers, and we're hiring for engineers too - get in touch if you're curious about any of the roles.

What should be logged?

The first logical question when tackling audit logs, is what actually should be logged, and what shouldn't. A good rule of thumb is if something changes in your database, it should be logged. Typically, this means all POST, UPDATE, PATCH and DELETE requests. GET requests typically would not be logged, but that doesn't mean they can't be if you've got a compelling reason to know when someone viewed something, for instance.

Ask yourself the question "As the customer managing the configuration or monitoring of this app, would I like to know when this activity has happened, by who, and when?". If the answer is a yes, then you need an audit log.

What should be in an audit log?

For me, there are a few key elements of an audit log that need to be there in order for it to be effective and useful to your team.

1. The entity that was changed. What was the entity that was created/updated/deleted etc. In the example below, the entity was a "resource".
2. The type of event. Was it an update, a deletion, or something else? Being able to glance at an audit log and know the action that was taken is essential.
3. When the event occurred. You need to know when the event happened, and of course if you're showing a list of events, they should be in chronological order. Bonus points here if you show both the time, and relative time (eg "10 minutes ago")
4. Who made the change. Knowing who made the change is essential. Was it a support team member, or the customer themselves? Bonus points here for including the user ID, to make chasing down the culprit that bit quicker.
5. What was changed? Understanding the properties that were changed, the previous value, and the new value, is essential. Without this information, your audit logs are a lot less effective.

How long should I keep audit logs?

This is up to your business, but in my experience, typically any more than 3 months is overkill. If there is anything to debug, it'll usually crop up in that time frame. The only caveat to this would be large customers who want to retain logs for longer. This is of course an opportunity to upsell those clients onto a longer log retention period 😉.

Apps that do audit logging well

Of course I'm going to say Flipdish first, with reservadesk.com as a close second!

Other apps I've noticed that do it well are Stripe, split.io and localize.biz - although in localize's case it's more of a light activity log.

Anything else I should know, Chris?

I'm glad you asked. Once you have your audit log system set up, and it has appropriate filters, you should link to the audit logs from every entity that has them. In our examples above, this means that if you're logged in as an organisation teammate, and you view a resource like Desk 1, you should have a link that goes directly to your audit logs list page that has the filter pre-filled for that resource.

Or even, better, display the logs for that item in the UI itself, no need to redirect the user.

Did you like this post? Subscribe for more tidbits in product development, things I've learned from Flipdish, and more. I'm @cderm on twitter and Chris Dermody on Linkedin