The Developer’s Guide to Directory Sync / SCIM

Signups are great, but your product only grows when your customers actually use it. Adding Directory Sync to your app can help improve activation rates and land those larger enterprise deals. Like SSO and SAML, implementing Directory Sync is full of archaic standards, versioning nightmares, and manual integrations; it can be a lot to handle. Our guide will walk you through everything Directory Sync: what it is, why it’s important, protocols like SCIM, and how to build it into your product.

The Basics: What Directory Sync is and why you should care

‍To understand why you’d want to integrate Directory Sync into your app, let’s think about things from your customer’s perspective.

Imagine you’re an enterprise – we’ll use Nike as an example. Nike has hundreds (probably thousands) of apps that their employees use, from internal tools to external SaaS like Salesforce and GitHub. If you’re an IT admin at Nike, you need to make sure that all 75K+ Nike employees have accounts in the apps they need to use, and that when those employees leave, you can revoke their access to those accounts. Add in the complex stuff like revoking session tokens and SSH keys, and it’s not hard to understand why this is a notoriously difficult job.

In the IT world, this is called User Lifecycle Management (ULM). Broadly speaking, it’s mostly about automatically provisioning and de-provisioning users when they join an organization, but that’s easier said than done. ULM is often manual and error prone, and it’s getting worse: organizations are using way more cloud-based tools than they used to. When you leave your job, your IT admin needs to worry about your accounts in GitHub, Netlify, AWS, Segment, Sentry... the list goes on. This is where Directory Sync comes in: it automates the lifecycle management process by providing a single source of truth for identity. The end product is a bunch of endpoints that give apps information on users.

Directory Sync is a single source of truth for identity

You’ve probably worked with an employee directory tool like Bamboo or Rippling before (peeking on who’s managing who): Directory Sync is pretty much the same thing, but built for programmatic access through a standard protocol. You define your users and groups, and standard endpoints let your applications work with that data so you don’t have to.

Let’s get back to our Nike and GitHub example. Internally, Nike has hundreds, if not thousands of developers; when each of those developers got hired, IT would need to manually create a GitHub account for them with the appropriate permissions (which repos can they contribute to? Which groups are they part of?). And if and when they leave Nike, those accounts need to get shut down. Implementing Directory Sync helps automate that process, which is why GitHub built a Directory Sync integration.

When you’re dealing with user lifecycle management, the devil is really in the details; creating and revoking account access is just the tip of the iceberg. A good example is existing authenticated sessions. You don’t actually have to log in to most of your cloud apps every time you use them, because your auth token (or however they implement it) gets refreshed automatically. IT admins need to worry about revoking those sessions once employees leave, or their accounts aren’t really locked down.

There are a bunch of protocols and services for setting up Directory Sync: the most popular (and open source) one is called SCIM.

SCIM: the System for Cross Identity Management

SCIM is an open source protocol for implementing Directory Sync: it sets up a central source of truth for your company’s identity data that can interact with your cloud apps via a REST API. You maintain a SCIM server (or any implementation), and third party apps talk to it to figure out which accounts to create and destroy, which permissions users should have, which groups they belong to, etc. Enterprise-ready apps like Salesforce, GitHub, and Dropbox all have native SCIM integrations.

Behind the scenes, SCIM is just a bunch of APIs that let you create and manipulate JSON. There are User and Group objects, each with standard metadata (IDs, timestamps, etc.) and custom defined fields like name, role, or whatever else you want to include. SCIM exposes that data through REST endpoints, which is how your applications interact with it. A few examples:

• Create a new user: POST https://scim-example.com/2.0/user
• Delete a user: DELETE https://scim-example.com/2.0/user
• Read all groups: GET https://scim-example.com/2.0/group

The payload for creating a new user would look something like this (from the SCIM site):

The specific fields in the request body depend on your implementation, but metadata like the schemas and externalId are part of the protocol.

If this seems basic, it’s because the value here isn’t necessarily in some elegant solution to a complex technical problem; it’s about creating a standard so that you don’t need to write custom integration code for every app. But you’d also be wrong, because working with SCIM is anything but basic; it’s actually quite difficult.

You might have noticed that the endpoints above had a version in them; the first implementation “red flag” is that there are two major SCIM versions – 2.0 and 1.1 – and they don’t work well together. Each Directory Sync provider also has different requirements (e.g. Google only lets you pull data and doesn’t push), which means adding support to your app is going to require manual configuration and updates. Some SCIM implementations literally require manual CSV uploads every hour. It’s rough out there, so make sure you do your research before building this yourself (more on that later).

Directory Sync vs. JIT provisioning: activating your users

Now that you have a better sense of what a Directory Sync protocol like SCIM actually does, it’s easier to understand why it can be such a powerful lever for growth. Essentially, it speeds up the time from account creation to users getting value out of your product. Instead of waiting for users to create their accounts, you can create them all automatically.

For another perspective, it’s worth talking about how companies use cloud apps without Directory Sync. How do you provision accounts for every user in your organization without a central source of truth? The current standard is called JIT (just in time) provisioning: it just means that you provision an account for the first time when a user with the right domain attempts to sign in (if you’re at Nike, it’s @nike.com). The account doesn’t exist until the user creates it, so the creation is happening... just in time.

This is where Directory Sync really shines. If you’re onboarding 1,000 new developers to GitHub, Directory Sync will create all of those accounts right away. They’ll already have their name, email, and profile picture populated; and you’ll be able to tag other developers in pull requests even if they haven’t signed in yet. This dramatically increases activation and internal virality because it removes a step of friction and puts users directly into the product.

Another important consideration for Directory Sync vs. JIT provisioning is access controls. Allowing just in time account creation based on email domain is great for accelerating internal virality – it was a critical feature that helped Slack grow so quickly – but it can lead to users getting access to things that they shouldn’t (think: an executives only Slack team). That’s why some IT admins today want to disable JIT, and Slack eventually had to add that capability to the product.

Getting practical: how to add Directory Sync to your app

You can build Directory Sync into your app yourself, or pay for someone else to take care of it for you. We’ll walk through the basics of both.

1. Build: Building Directory Sync From Scratch

To get Directory Sync working with your app, you’ll need to build out a system that works with provided endpoints to provision and de-provision users, as well as take care of more advanced stuff like revoking session tokens:

• Add and remove accounts automatically
• Create and manage permission groups
• Update attributes and account settings

If you have existing endpoints for these tasks, your work will involve tying them to whichever provider endpoints give you the user and group data you need. If not, you’ll probably need to create them from scratch.

Each app is going to have its own unique implementation quirks. GitHub lets users authenticate via SSH keys in addition to username and password: when they built their Directory Sync integration, they needed to add functionality for disabling those keys. Dropbox lets you sync files to your desktop: their SCIM integration needed to account for how to remove those files. Your app is probably going to have one or two of these cases to account for too.

The hardest part of building Directory Sync yourself is supporting multiple providers. SCIM is one standard, but it’s not the only one (thanks xkcd) – companies also use providers like Azure Active Directory, GSuite Directory, LDAP, and Workday as sources of truth for Directory Sync. Depending on which deals you’re trying close, you’ll need to build custom connectors for those providers and their idiosyncratic APIs.

In short, building Directory Sync yourself is a lot like building SSO/SAML yourself. It’s not a technically complex problem; it’s a tedious and manual one that will require a ton of ongoing maintenance. It’s also something that can have catastrophic consequences if done poorly – onboarding a new organization with hundreds or thousands of users can take down your application if done incorrectly.

2. Buy: Directory Sync as a service provider

If you want to avoid this mess, you can outsource Directory Sync integration to a third party. Here's some solutions we found:

➞ Aquera

Aquera gives you out of the box connectors for SCIM and other directory providers. They also integrate natively with Okta. There’s no information about pricing on the site.

➞ Okta

If you’re already using Okta as an identity provider, the product provides a few integrations that let you take advantage of SCIM – essentially you can share Okta data with SCIM endpoints to provision users, among other things. Okta won’t exactly build Directory Sync for you, but it can help tackle a few of the tasks you’ll need to do to make that happen.

➞ WorkOS

WorkOS lets you add support for Directory Sync to your app with just a few lines of code, with support for SCIM, Okta, Azure AD, and most other popular directory providers.

What to do next

If you’re convinced that it’s time to add Directory Sync to your app (you should be), your first decision needs to be the how. Building it yourself will give you more flexibility for customization, but outsourcing to a third party provider will save engineering time and headaches down the road. Here are a few resources if you want to dig deeper:

• For more details on the SCIM spec: SCIM Docs
• For more info on GitHub’s SCIM integration: GitHub SCIM Docs
• For more details on Okta’s SCIM integration: Okta SCIM Docs
• For more detail's on Microsoft's Azure AD integration: Microsoft SCIM Docs