Sync data using the events API

In this guide, we will walk you through what you will need to integrate with the events API:

• Create a cursor for use with the events API
• Update your cursor
• Choose a cursor if you lose yours
• Handle event replay in your app

To get the most out of this guide, you’ll need:

• A WorkOS account
• An SSO or directory connection configured in order to generate events

WorkOS offers native SDKs in several popular programming languages. Choose a language below to see instructions in your app’s language.

To make calls to WorkOS, provide the API key and, in some cases, the client ID. Store these values as managed secrets, such as WORKOS_API_KEY and WORKOS_CLIENT_ID, and pass them to the SDKs either as environment variables or directly in your app’s configuration based on your preferences.

WORKOS_API_KEY='sk_example_123456789'
WORKOS_CLIENT_ID='client_123456789'

Your app can start consuming events once it integrates the WorkOS SDK. The first thing to do is to pick a starting place in the data set.

A cursor is a bookmark to track your app’s position in the events list. The very first call to the events API won’t have a cursor. Subsequent requests to WorkOS should include the updated cursor using the after parameter. You will need to update and store your cursor after processing an event.

To avoid repeating an update, store the updated_at timestamp provided by WorkOS for each object. Extract this tag from the data object in the event. If the updated_at timestamp in the event is newer, update the local state with the latest event data. Otherwise, you can skip processing that event.

Determine the event types you want to consume. Choose the relevant event types that align with your app’s functionality and integration with WorkOS.

import { WorkOS } from '@workos-inc/node';

const workos = new WorkOS(process.env.WORKOS_API_KEY);

const listOfEvents = await workos.events.listEvents({
  events: [
    'dsync.activated',
    'dsync.deleted',
    'dsync.user.created',
    'dsync.user.updated',
    'dsync.user.deleted',
  ],
});

Retrieve events from the API using cursor pagination. To fetch the next set of events, provide the ID of the latest processed event in the after parameter.

In some cases, it may be necessary to go back in time and “replay” the events. When designing your app logic to handle events replay it is important to design your event handling logic in a way that can safely accommodate it without undesired effects.

To achieve this, focus on separating your app’s data handlers from transactional business logic like sending emails, communicating to 3rd party APIs, etc. By implementing separate data handling, you can replay events without side effects.

WorkOS recommends processing events synchronously, handling them in the order they occurred.

When resuming event processing, you have two options to pick up where you left off:

• Using the latest known cursor: Retrieve the most recent cursor that was successfully processed and use it as the starting point for event resumption.
• Using a timestamp: Alternatively, you can make an API call with the range_start parameter and then use the cursor. Utilize the updated_at timestamp to prevent overwrites.

WorkOS recommends that your app starts with a single worker. Process events in a separate thread or process from your app’s main execution thread. Deploying a single worker responsible for handling events simplifies and streamlines event consumption.

This approach ensures serial event processing. After completing the processing of events on the current page, request the next page of events to maintain progress.

For onboarding large organizations, divide events into independent queues for parallel processing when calling the events API from a single worker.

Concurrently processing events from different organizations is safe, but for consistency and integrity, it is recommended to sequentially process events within a single organization.

You can migrate to the events API if you already use webhooks. To migrate, start querying the events API using the range_start parameter that corresponds to the time you’d want to start syncing from. The event IDs passed in webhook bodies are the same as those returned by the events API.