Best Practices for Audit Logs in Cloud Applications

In the ever-evolving landscape of application security, the ability to track who did what within your application and why they were allowed (Or forbidden) to do so is essential. This goal can be achieved by the use of audit logs. These records play a vital role in capturing and documenting a system's activities over time - which is especially crucial when dealing with application-level authorization.

This article will discuss the importance of audit logs, specifically their role in tracking authorization-related events, the value of having “Meta” audit logs for authorization, implementation methods, audit log generation best practices, and the importance of having comprehensible audit logs to adhere to compliance standards.

What Are Audit Logs?

An audit log is a central record of events impacting a system's state or behavior. Each log entry acts as a footprint, detailing a specific change in the system. It is usually accompanied by contextual information about its initiation and, where applicable, the reasons behind it.

In the context of application-level authorization, an audit log will usually include information such as:

• A timestamp of the attempted action

• The user/machine who attempted the action

• A description of the attempted action

• The resource which was the target of the attempted action

• The authorization decision - Permitted or Denied,
  or the status of the operation - Success or Failure.

• And, in some cases -
  The reasoning behind the authorization decision

Authorization Audit-Logs in the Permit.io dashboard

{
      "Allow": true,
      "debug": {
         "rbac": {
           "code":
           "allow": true,

      "reason":"user 'user_1' has the role 'admin' in tenant,
               'sample_tenant', role 'admin' has the 'create' permission 
               on resources of type 'document'"

           "allowing_roles": [
             "admin"
           ]
      }, 
      "request": {
         "user": {
             "key": "user_1",
             "email": null,
             "synced": true,
             "last_name": null,
             "attributes": {
               "age": 50,
               "key": "user_1",
               "email": "sampleemail@org.com",
               "roles": [ 
                 "admin"
               ],             
               "tenants": [
                 "sample_tenant"
               ],             
               "fav_color": "red"
             },
             "first_name": null
           },
           "action": "create",
           "tenant": "sample_tenant",
           "resource": {
             "type": "document",           
             "attributes": {
               "type": "document"
             }
           }
         }
       },

      "allowing_sources": [
         "rbac"
      ],
      "debugger_activated": true,
      "__data_use_debugger": true,
      "__input_use_debugger": null   
   }

An example of a Permit.io-generated authorization audit log for an RBAC policy

This information will allow an administrator to easily understand and investigate the reasoning behind authorization decisions made within their system. Having detailed audit logs is a crucial function, especially when investigating unauthorized access. Without them, there would be no way to trace the origin and reasoning behind problematic actions that occurred in our application.

Audit logs are our best way of debugging the authorization system itself, debugging behaviors in the access-controlled application, understanding the cascading results of authentication mechanisms, and, of course, investigating security/compliance incidents.

Taking it one step further - it is extremely important to have access not only to audit logs of actions made in the application itself, but also to changes made within the application’s access control layer. Let’s dive a little deeper into that.

Why Do You Need “Meta” Audit Logs?

While having a detailed log of all the authorization events happening in your application is crucial, it’s not enough. Understanding the changes made to your application’s authorization layer is another important step toward a fully transparent view of your application’s access control.

This extra level of auditing (Which we refer to here as “Meta” audit logs) tackles one of the most complex issues in the world of access control - Authorization-for-Authorization. It provides insights into the granting, changing, and revoking of permissions within the application's authorization system. By capturing these events, organizations gain visibility into the evolution of permission assignments, ensuring transparency and accountability. “Meta” audit logs enable administrators to monitor and review changes to permissions, identify potential misconfigurations or unauthorized access, and strengthen their overall security posture.

An example of “Meta” Audit Logs generated via the Permit.io UI

How to Properly Implement Audit Logs for Authorization?

To effectively implement audit logs in your application, it is crucial to record each significant event that occurs. At its core, an audit log is a repository, either a file, database table, or persistent storage, where a new record is added for every event. When generating an audit event, capture relevant system information and label it with the corresponding event type.

Setting up a dedicated log viewer offers convenient access to your records. There are several options available for indexing and consuming audit events - as you probably won’t want to do it manually:

• Streaming audit logs to stdout: Log aggregation tools, such as Elastic, can collect logs directly from your application's standard output stream, particularly when you use formats like JSON with a consistent structure.

• Viewing logs: Tools like Datadog provide comprehensive features for aggregating, browsing, and searching log data. Once you have ingested the audit logs from your system, these applications allow you to query events based on properties such as type and timespan.

• Collecting logs with Logstash: As part of the Elastic suite, Logstash serves as a versatile data collector capable of reading logs from various sources. It can retrieve logs from local files, events in a database, and requests sent to an HTTP webhook or websocket. Once the data is ingested, it is forwarded to your preferred log aggregator.

• Using an authorization solution that provides built-in audit log aggregation (such as Permit.io) in addition to log streaming.

• No matter which implementation method you choose, make sure it is reliable:

Your chosen processing pipeline must be capable of handling audit logs reliably. Although occasional drops in error logs might be acceptable, a failed audit log write could have serious consequences for future compliance investigations.

Generating Audit Logs: Best Practices

Having understood the different approaches to implementing an audit solution, let’s zoom in on the generation of the audit logs themselves. The best practices for how audit logs should be generated can be as important as their implementation, if not more so.

As we generate audit logs for multiple users, tenants, and applications, we have compiled a list of best practices for making your logs more helpful and coherent:

• Automatic Generation:
  Generate audit logs from access checks automatically, and avoid having to report each audit log individually. This will grant your audit logs full coverage, preventing any potential gaps. If discovered post-incident, these gaps might be too late to fill.

• Asynchronous Audit Logging:
  Audit but logs can be very noisy and uneasy to investigate. It is recommended to use dedicated functions that record each action asynchronously, without adding latency to each route. This approach helps ensure a smooth and efficient investigation process.

• Generate Audit Logs Implicitly:
  It is recommended to Prefer solutions that generate audit logs implicitly, without the need for explicit logging code in your application. This allows for better consistency, more comprehensive coverage, reduced code complexity, and better log standardization.

• Include decision logs:
  Prefer audit logs that include decisions logs (Logs that refer to the state of the decision engine and the specific logic that led to an authorization decision). This will allow you to better replicate the flow that created the audit log when investigating or debugging.

• Prefer annotated audit logs :
  Including annotations in audit logs provides valuable information and insights into the context of the system, the reasoning behind events, and why decisions were approved or denied.

Abiding by these practices can help enable efficient investigation and debugging processes, provide valuable insights into the context and reasoning behind authorization events, and facilitate adherence to compliance standards.

Let’s dig a little deeper into the compliance aspect -

How to Use Audit Logs to Adhere to Compliance Standards?

Having accessible audit logs for your application’s authorization layer is a must for maintaining compliance standards within an organization.

Compliance standards, such as HIPPA, SOC2, and ISO 27001 require organizations to implement robust access controls and auditing mechanisms. Authorization layer audit logs are instrumental in ensuring that only authorized personnel access customer data by providing visibility into user access activities and detecting any deviations from the defined access controls. By maintaining comprehensive audit logs, organizations can demonstrate their adherence to HIPPA, SOC2, and ISO 27001, enhance data protection measures, and instill confidence in their clients and stakeholders regarding the security and privacy of sensitive information.

Audit Logs for Different Policy Models, Engines, and Languages

Authorization is never as simple as “One solution fits all” - it's essential to explore audit logs in the context of different policy engines and access control models.

Audit Logs for RBAC (Role-Based Access Control):

⁠Role-Based Access Control (RBAC) is a widely adopted model that assigns permissions based on predefined roles. Having audit logs specifically tailored for RBAC enables organizations to effectively monitor and trace access control decisions. These logs capture information about user activity based on their assigned roles and provide insight into who accessed what and when.

Audit Logs for ABAC (Attribute-Based Access Control):

Attribute-Based Access Control (ABAC) is a flexible access control model that grants permissions based on attributes associated with users, resources, and environmental conditions. In an ABAC environment, audit logs provide granular details about attribute evaluations, policy decisions, and access requests. By capturing the attributes considered during authorization decisions, ABAC audit logs facilitate comprehensive auditing and analysis.

Audit Logs for ReBAC (Relationship-Based Access Control):

⁠Relationship-Based Access Control (ReBAC) extends the traditional RBAC model by considering relationships between users, resources, and contexts when making access control decisions. Audit logs tailored for ReBAC capture crucial information about these relationships, allowing developers to understand the complex relationships influencing authorization outcomes.

Audit logs play an increasingly important role in different policy engines and languages:

Audit Logs for Open Policy Agent (OPA):

Open Policy Agent (OPA) is a popular policy engine used for fine-grained authorization. OPA provides a flexible and extensible framework for expressing and enforcing access control policies in the Rego policy language. Audit logs specific to OPA capture decision evaluations, policy matches, and enforcement actions. These logs enable developers to monitor and review policy decisions made by OPA, helping to ensure compliance and identify policy gaps or misconfigurations.

Audit Logs for AWS Cedar:

AWS’ Cedar is a new policy engine and language that enables developers to define and enforce complex authorization policies. Cedar-specific audit logs track policy evaluations allowing developers to gain insights into the enforcement of intricate authorization policies, detect anomalies, and verify compliance with desired access control outcomes.

Ready-to-Use Audit Logs:

To effectively leverage audit logs for application-level authorization, Permit.io offers automatically generated audit logs out of the box for both your authorization layer (Supporting RBAC, ABAC, and ReBAC policy models, and both the OPA and Cedar engines), as well as your authorization layer management. While Permit’s UI provides a dedicated page for viewing your audit logs, you can also find them directly in your deployed PDP microservice. As they are written to its stdout/stderr, you can stream them directly into your logging platform.

Permit’s audit log interface also allows you to filter them based on various aspects, such as users, dates, decisions, and more.

An example of Permit’s Audit log UI filtering mechanism

Conclusion:

In the realm of application security, audit logs play an integral role in application-level authorization. Capturing a comprehensive set of audit logs, including meta-audit logs that encompass permission management activities, is essential to maintaining a secure and compliant environment. With Permit.io's integrated authorization solution and its ready-to-use audit logs, you can easily streamline your security operations, improve compliance, and fortify your defenses against unauthorized access.

Want to learn more about Authorization? Join our Slack community, where there are hundreds of devs building and implementing authorization.