The Developer’s Guide to Audit Logs / SIEM

Building audit logs into your app will help you land larger deals and give your customers confidence in your product’s security profile—but actually choosing which events to log, determining which payloads to include, and exposing events via the right APIs is far from easy. 

Our guide will walk you through: 

• What audit logs are and why they are important
• How to structure your audit logs
• Strategies for storing logs efficiently
• Choosing the right SIEM tools to manage and analyze your audit logs

Let’s start by explaining audit logs and why they are important.

What are audit logs and why should you care?

Security officers, compliance managers, and IT staff need full visibility into the apps their teams are using, including who’s logging in, resetting their passwords, and accessing what data. 

Let’s imagine we’re an internal IT admin at Kellogg’s — and we want visibility into how our organization uses Salesforce. A few things we might be curious about:

• Has someone tried and failed to log in 10 times in the past hour?
• Who accessed PII (personally identifiable information) in the past week?
• How many users accessed a sensitive report accidentally shared with the wrong team?

Audit logs answer these questions by providing a granular paper trail of app users' actions. Every time a user does anything of consequence—log in, reset the password, create a report, you name it—your system emits an event with useful metadata. Companies will expose these events via a UI and an API so admins can analyze them at scale.

Use cases of audit logs

The data in audit logs have various possible use cases for IT admins.

• Monitoring and alerts: Admins primarily use audit logs to inspect user behavior during configuration and set up alerts for potential malicious actors or compromised accounts.
• Compliance with regulations: In specific industries such as finance, some laws require companies to store a “paper trail” of user behavior (such as messages) in an audit log, often called data retention. 
• Digital forensics: These logs may also be used for digital forensics in the case of a breach or “e-discovery” during a lawsuit. 

Audit logs demonstrate a robust security posture for service providers. They can help your business move upmarket, improve retention, and close existing target deals faster.

Building audit logs into your app

Building audit logs into your app isn’t technically difficult, particularly if you have experience with tools like Segment, as the coding process is similar.

However, as with most enterprise features, the devil is in the details. You’ll need to decide:

• What kinds of events to log
• What the payload should be
• How long should you store audit logs
• What frontend to build
• How to integrate these events with the broader enterprise toolkit, e.g., SIEM tools

What kind of events to log

The goal of audit logs is to provide a paper trail of important actions that help IT admins and security folks get a detailed picture of what’s happening in your app—that’s the driving logic behind what kinds of events you should be writing. 

Unlike analytics or observability events, audit logs should capture:

• Login/auth: Events like user registration, login, password resets, and username updates.
• Data access: Actions such as opening, modifying, or deleting documents.
• Billing: Updates to billing or subscription details.
• Search/navigation: Activities related to using search functions or navigating within the app.

Conversely, here are a few examples of things that you shouldn’t be including in your audit logs:

• Performance data: Metrics like runtime duration or CPU usage.
• Internal tracing: Error logs, exceptions, and stack traces.

If you’re not sure what kind of events to include, look at other products and talk to your customers—they might have specific requirements custom-made to their compliance needs.

What the payload should be

There’s no widely accepted standard for audit log payloads yet, so things are a mess. The general rule of thumb is to include enough information to be helpful but nothing that doesn’t fit the audit log use case (e.g.,d product data). 

The typical format looks something like this:

• An actor: The ID of the user or system acting.
• A group: The user’s permission group(s), usually an organization ID or domain.
• An action type: The Event name (passwordchanged, authenticationsuccessful).
• Timestamp: A timestamp in UTC to maintain consistency.

Some organizations use the Common Event Format (CEF) from ArcSight, standardizing fields for easier security systems integration. However, in practice, most audit log systems do not match the CEF.

Whether you follow the CEF or not, standardize internally. Keeping the same structure across all audit events will improve your user experience and simplify building your audit logs portal.‍

How long should you store audit logs?

If you’re supporting thousands of users, your audit logs will take up a lot of space. Companies typically run a 90-day retention window and allow customers to pay for longer-term storage, such as 1 - 3 years. They store these logs in relational stores like Postgres, search systems like Elasticsearch, and even as flat files in an S3 bucket. 

Note that you’ll need to index the logs for fast search and filtering and make sure the logs are immutable once they’re stored, as that’s the whole point.

What frontend to build 

Once you’ve set up the right system for emitting events on the backend, you’ll need to build a frontend to expose those events to your users. We’ve seen implementations as simple as a table of events and as full-featured admin panels with search, filtering, sorting, and exporting to SIEM solutions (more on that later).

What you build depends on the use cases your customers have. Detailed frontends are useful if your users are going to interact with audit logs directly in your app. Still, they can be overkill if you’re dealing with much larger customers who will want to export these logs to a different tool anyway.

Once you’ve got your events system and frontend in check, you’ve got one more thing to worry about — integration.

Integrating with SIEM tools

When you’re serving hundreds or thousands of users, combing through audit logs manually to find the data you need isn’t going to work. Enterprise admins use specialized tools called SIEM (Security Information and Event Management) to ingest audit logs and provide search, grouping, and visualization.

One of the best-known SIEM tools is Splunk. It’s a comprehensive platform designed for security event aggregation, data analysis, and visualization tool with features built specifically for audit logs. 

Another good example is Sentinel, an Azure-specific SIEM from Microsoft. It offers a dashboard that aggregates key events over time, as well as granular search and filtering for when you need to dive into specific patterns.

Enterprises will typically use an SIEM tool, so your systems are going to need to integrate and export data, which, as you can probably guess, isn’t very much fun. 

Each SIEM solution is going to have different ingestion requirements. Currently, there are at least 35 popular SIEM providers out there, and many large enterprise admins have bespoke solutions, too. You’re lucky if they only request to drop logs in an S3 bucket. We’ve heard of IT admins asking to stream logs through an SSH tunnel.

Walkthrough example: Slack

To put all of this together, let’s take a look at a company that has successfully moved upmarket by building great enterprise-ready features — Slack. If you’re on the Enterprise Grid plan, Slack provides access to audit logs via a read-only REST API that covers pretty much any event that would be of interest to an IT admin or security team. 

Here are a few examples of the kinds of audit events that Slack retains:

• workspace_created: Emitted when a new workspace is created.
• organization_created: Emitted when a new organization is created.
• ekm_enrolled: Emitted when an organization enrolls in Slack’s EKM.
• member_joined_channel: Emitted when a user joins a channel.

Each event comes with a semi-standard payload with details like the actor (who did it), the action (what happened), and the context.

This API makes integrating with SIEM providers just a bit easier. Their docs also cover how to install external apps that interact with the audit logs API.

Beyond the REST API, Slack also provides a basic frontend for interacting with audit logs. On smaller plans, you just see a list of access logs:

If you’re curious about how Slack started landing larger deals via enterprise features, check out our profile here.

Adding audit logs to your app

Ready to add audit logs to your app? Use WorkOS Audit Logs and integrate customizable, exportable, and streamable audit logs into your app in just a few lines of code.

• Getting started is easy: With easy-to-use SDKs, you can quickly start sending event data from your app to the WorkOS Audit Log API.
• Customizable and exportable: Define the specific events, actions, and metadata that matter most to your app directly from the WorkOS dashboard.
• Integration with SIEM tools: Effortlessly stream audit logs to popular SIEM tools like Datadog, Splunk, or AWS S3. The best part? They can set it up themselves with their existing SIEM provider directly from the WorkOS Admin Portal.
• Generic log streams: Stream logs to any endpoint that accepts HTTP post requests with a JSON payload.
• Transparent pricing: WorkOS offers straightforward pricing, starting at $5 per month for each organization you onboard.

Sign up for WorkOS today, and start selling to enterprise customers tomorrow.