The Developer’s Guide to Directory Sync / SCIM

Signups are great, but your product only grows when your customers actually use it. Adding Directory Sync to your app can help improve activation rates and land those larger enterprise deals. 

Like SSO, implementing Directory Sync is full of archaic standards, versioning nightmares, and manual integrations; it can be a lot to handle. 

In this guide, we will walk you through:

• What WorkOS Directory Sync is and why it’s important
• Using SCIM to implement Directory Sync
• How to build Directory Sync into your product

Let’s start with the basics of Directory Sync.

What is Directory Sync is and why you should care?

‍To understand why you should integrate Directory Sync into your app, consider your customer’s perspective.

Imagine you’re an enterprise — we’ll use Nike as an example. Nike has hundreds (probably thousands) of apps that their employees use, from internal tools to external SaaS apps like Salesforce and GitHub. If you’re an IT admin at Nike, you need to make sure that all 75,000+ Nike employees have accounts in the apps they need to use and that when those employees leave, you can revoke their access to those accounts. 

Add in the complex stuff like revoking session tokens and SSH keys, and it’s not hard to understand why this is a notoriously difficult job.

In the IT world, this is called User Lifecycle Management (ULM). It’s mostly about automatically provisioning and de-provisioning users when they join an organization, but that’s easier said than done. ULM is often manual and error-prone, and it’s getting worse. Organizations are using way more cloud-based tools than they used to. 

When you leave your job, your IT admin needs to worry about your accounts in GitHub, Netlify, AWS, Segment, Sentry… the list goes on. This is where Directory Sync comes in: it automates the lifecycle management process by providing a single source of truth for identity. The end product is a bunch of endpoints that give apps.

Directory Sync is a single source of truth for identity

You’ve probably worked with an employee directory tool like Bamboo or Rippling before (peeking on who’s managing who): Directory Sync is pretty much the same thing, but built for programmatic access through a standard protocol. You define your users and groups, and standard endpoints let your applications work with that data so you don’t have to.

Let’s get back to our Nike and GitHub example. Internally, Nike has hundreds, if not thousands of developers; when each of those developers got hired, IT would need to manually create a GitHub account for them with the appropriate permissions (which repos can they contribute to? Which groups are they part of?). And if and when they leave Nike, those accounts need to get shut down. Implementing Directory Sync helps automate that process, which is why GitHub built a Directory Sync integration.

When you’re dealing with user lifecycle management, the devil is really in the details; creating and revoking account access is just the tip of the iceberg. A good example is existing authenticated sessions. You don’t actually have to log in to most of your cloud apps every time you use them, because your auth token (or however they implement it) gets refreshed automatically. IT admins need to worry about revoking those sessions once employees leave, or their accounts aren’t really locked down.

There are a bunch of protocols and services for setting up Directory Sync: the most popular (and open source) one is called SCIM.

SCIM: the System for Cross Identity Management

SCIM is an open source protocol for implementing Directory Sync: it sets up a central source of truth for your company’s identity data that can interact with your cloud apps via a REST API. You maintain a SCIM server (or any implementation), and third party apps talk to it to figure out which accounts to create and destroy, which permissions users should have, which groups they belong to, etc. Enterprise-ready apps like Salesforce, GitHub, and Dropbox all have native SCIM integrations.

Behind the scenes, SCIM is just a bunch of APIs that let you create and manipulate JSON. There are User and Group objects, each with standard metadata (IDs, timestamps, etc.) and custom defined fields like name, role, or whatever else you want to include. SCIM exposes that data through REST endpoints, which is how your applications interact with it. A few examples:

• Create a new user: POST https://scim-example.com/2.0/user
• Delete a user: DELETE https://scim-example.com/2.0/user
• Read all groups: GET https://scim-example.com/2.0/group

The payload for creating a new user would look something like this (from the SCIM site):

	
POST /v2/Users  HTTP/1.1
Accept: application/json
Authorization: Bearer h480djs93hd8
Host: example.com
Content-Length: ...
Content-Type: application/json

{
  "schemas":["urn:ietf:params:scim:schemas:core:2.0:User"],
  "externalId":"dschrute",
  "userName":"dschrute",
  "name":{
    "familyName":"Schrute",
    "givenName":"Dwight"
  }
}
	

The specific fields in the request body depend on your implementation, but metadata like the schemas and externalId are part of the protocol.

If this seems basic, it’s because the value here isn’t necessarily in some elegant solution to a complex technical problem; it’s about creating a standard so that you don’t need to write custom integration code for every app. But you’d also be wrong, because working with SCIM is anything but basic; it’s actually quite difficult.

For example, you might have noticed that the endpoints above include version numbers. SCIM has two major versions — 1.1 and 2.0 — that are not fully compatible, which is one of the first hurdles in implementation if you need to support both. Each Directory Sync provider also has different requirements (e.g., Google only lets you pull data and doesn’t push), which means adding support to your app is going to require manual configuration and updates. Some non-standard SCIM implementations literally require manual CSV uploads every hour. 

It’s rough out there, so make sure you do your research before building this yourself (more on that later).

Directory Sync vs. JIT provisioning: activating your users

Now that you have a better sense of what a Directory Sync protocol like SCIM actually does, it’s easier to understand why it can be such a powerful lever for growth. Essentially, it speeds up the time from account creation to users getting value out of your product. Instead of waiting for users to create their accounts, you can create them all automatically.

For another perspective, it’s worth talking about how companies use cloud apps without Directory Sync. How do you provision accounts for every user in your organization without a central source of truth? The current standard is called JIT (just in time) provisioning: it just means that you provision an account for the first time when a user with the right domain attempts to sign in (if you’re at Nike, it’s @nike.com). The account doesn’t exist until the user creates it, so the creation is happening... just in time.

This is where Directory Sync really shines. If you’re onboarding 1,000 new developers to GitHub, Directory Sync will create all of those accounts right away. They’ll already have their name, email, and profile picture populated; and you’ll be able to tag other developers in pull requests even if they haven’t signed in yet. This dramatically increases activation and internal virality because it removes a step of friction and puts users directly into the product.

Another important consideration for Directory Sync vs. JIT provisioning is access controls. Allowing just in time account creation based on email domain is great for accelerating internal virality – it was a critical feature that helped Slack grow so quickly – but it can lead to users getting access to things that they shouldn’t (think: an executives only Slack team). That’s why some IT admins today want to disable JIT, and Slack eventually had to add that capability to the product.

How to add Directory Sync to your app from scratch

To get Directory Sync working with your app, you’ll need to build out a system that works with provided endpoints to provision and de-provision users, as well as take care of more advanced stuff like revoking session tokens:

• Add and remove accounts automatically
• Create and manage permission groups
• Update attributes and account settings

If you have existing endpoints for these tasks, your work will involve tying them to whichever provider endpoints give you the user and group data you need. If not, you’ll probably need to create them from scratch.

Each app is going to have its own unique implementation quirks. GitHub lets users authenticate via SSH keys in addition to username and password: when they built their Directory Sync integration, they needed to add functionality for disabling those keys. Dropbox lets you sync files to your desktop: their SCIM integration needed to account for how to remove those files. Your app is probably going to have one or two of these cases to account for too.

The hardest part of building Directory Sync yourself is supporting multiple providers. SCIM is one standard, but it’s not the only one (thanks xkcd) – companies also use providers like Azure Active Directory, GSuite Directory, LDAP, and Workday as sources of truth for Directory Sync. Depending on which deals you’re trying close, you’ll need to build custom connectors for those providers and their idiosyncratic APIs.

The development and onboarding process for Directory Sync is not just complex; it's intricate and requires a detailed understanding of various directory providers. It's generally more complex and time-consuming than SSO. It’s also something that can have catastrophic consequences if done poorly – onboarding a new organization with hundreds or thousands of users can take down your application if done incorrectly.

How to add Directory Sync to your app using a provider

If you want to avoid this mess, you can outsource Directory Sync integration to a third party. In this section we will see some of the available solutions.

Aquera

Aquera gives you out of the box connectors for SCIM and other directory providers. They also integrate natively with Okta. There’s no information about pricing on the site.

Okta

If you’re already using Okta as an identity provider, the product provides a few integrations that let you take advantage of SCIM – essentially you can share Okta data with SCIM endpoints to provision users, among other things. Okta won’t exactly build Directory Sync for you, but it can help tackle a few of the tasks you’ll need to do to make that happen.

WorkOS

WorkOS lets you add support for Directory Sync to your app with just a few lines of code. It's one of the best SCIM connectors out there for a few reasons:

• Getting started is fast: With SDKs for every popular platform and Slack-based support, you can implement Directory Sync in minutes rather than weeks.
• Unified SCIM integration: WorkOS’s Directory Sync API integrates with over a dozen major identity providers, including Microsoft Entra, Okta, Workday, and any other SCIM-compliant directories. You can support any enterprise customer out of the box without creating custom connectors for each.
• Events-based processing: While webhooks are also supported, WorkOS’s Events API means every SCIM request is processed in order and in real-time. You’ll never miss a provisioning request again.
• Support for custom attributes: Many enterprise customers need custom attributes beyond basic user profiles — such as employee numbers, departments, and cost centers. WorkOS allows you to add custom attributes specific to each organization’s needs.
• Simplified onboarding: WorkOS’s Admin Portal takes the pain out of onboarding your customers’ IT teams and configuring your app to work with their identity provider. Your customers can set up and configure Directory Sync on their own without requiring extensive back-and-forth with your team. 
• Pricing that makes sense: Unlike competitors who price by monthly active users, WorkOS charges a flat rate for each company you onboard — whether they’re syncing 10 or 10,000 users with your app.

What to do next

If you’re convinced that it’s time to add Directory Sync to your app (you should be), your first decision needs to be the how. In most cases outsourcing to a third party provider is the way to go. It will save engineering time and headaches down the road.

To learn more about the challenges SCIM poses, see the following:

• Build vs buy part I: complexities of building SSO and SCIM in-house
• SCIM challenges: navigating the idiosyncrasies of different providers
• Implementation challenges of a homegrown SCIM solution
• SCIM complexity explained: Tackle group fragmentation