SCIM challenges: navigating the idiosyncrasies of different providers

SCIM is a spec designed to simplify identity management. It offers methods for creating, reading, updating, and deleting user accounts across multiple systems, facilitating seamless identity data integration between different applications and directories.

However, since SCIM is a spec and not a strictly implemented protocol, individual providers may interpret and implement it in slightly varied ways or introduce ambiguous scenarios that the spec does not adequately address. Building this in-house means you must deal with fragmentation issues across onboarding, implementation, and triage, incurring significant engineering costs, delayed time to market, and potential security issues.

Not addressing each provider’s idiosyncrasies can cause downstream applications to interpret data from different providers inconsistently, suffer from data duplication (resulting in HTTP 409 errors), or even require tailored solutions for specific actions (e.g., having to remove users manually before deleting a group due to a lack of group delete SCIM requests).

The development and onboarding process for SCIM is not just complex; it's intricate and requires a detailed understanding of various directory providers. It's generally more complex and time-consuming than SSO. In this article, we will see some of the reasons why.

Data fragmentation

A user's email address is critical information needed for a successful SCIM integration. Unfortunately, this field is not mandatory in many providers, which can lead to invalid events. This is just one example of data fragmentation that your application must solve.

Another common use case is adding custom attributes to a user profile in your application, like a user’s job title or profile picture. This is handled differently across IdPs:

• Entra ID (Azure) requires adding proper schema extension as prefix urn:ietf:params:scim:schemas:extension:enterprise:2.0:User or urn:ietf:params:scim:schemas:core:2.0:User for the attribute to show up, and the attributes will always come in as nested.
• In Okta attributes with urn:ietf:params:scim:schemas:core:2.0:User prefix do show up as top-level attributes.
• Rippling, allows only a limited set of custom fields in SCIM applications.

The more providers you support, the worse the data fragmentation gets.

User deactivation fragmentation

Providers handle user deactivation and reactivation differently. Some providers will delete the user, others will only suspend the user if they’re not part of a group, and others will send a request to set the user as no longer active. Worse still, the user’s permissions and access rights may remain active when they should not, and these changes may not be communicated to your app.

Some examples:

• The SCIM RFC specifies a DELETE endpoint for removing a resource. But Okta instead sends a PUT/PATCH request to set a user’s status to inactive. When it comes to suspension in Okta, it will only affect the user’s login and will not alter their status in any connected applications.
• Entra ID (Azure) does use the DELETE endpoint.

With different IdPs handling suspension differently, you’ll have to process requests with unique logic and flows according to each provider. You can imagine what happens when you’re dealing with 10-20 providers.

Group management fragmentation

Access management is handled via group memberships: for example, if you’re in the “engineering” group, you get write access to GitHub repositories. The problem is that every IdP handles group memberships slightly differently. For example, imagine you go on sabbatical, and get promoted while you’re away from engineer to engineering manager. You’d expect the IdP to issue a request that says that you’re no longer part of the engineering group. However, IdPs handle this differently:

• Okta issues a request that says that you’re no longer part of the engineering group, since you’re on sabbatical. So when you come back, you may erroneously still have access to engineering resources.
• Entra ID (Azure) does send a request in this situation.

Things get even more messy when we consider user deactivation, suspension, and reactivation scenarios. Due to the inconsistencies on how different IdPs handle them, a user’s memberships may remain mutable, though these changes are not consistently communicated. Here are just some of these nuances among popular SCIM providers:

• In Okta, if a user is suspended (rendered inactive) and their group memberships change during their suspension, Okta doesn’t communicate these changes to your app if the same group is used for assignment and push. As a result, when the suspended user is eventually reactivated, the changes in group memberships will not be reflected in your app, and they may end up having access permissions they shouldn’t.
• In Microsoft Entra ID, a user account is not suspended if the user is still a member of any provisioned groups. Users may retain access permissions they shouldn’t have. This means that to suspend a user, you should first remove them from any groups. And, of course, add them again if the user gets reactivated.
• Users who are removed from OneLogin are deleted rather than suspended. This means they are treated as new users if they are reactivated later. You’ll have to provision a new account for the user—any permissions or access rights associated with their previous accounts will have to be re-provisioned, which, if not done correctly, can cause users to end up with excess or limited permissions.
• When a group is deleted in JumpCloud, all users in the group will be disabled unless they’re associated with another active group. Users can retain permissions they shouldn’t have because they’re still active in another group. To completely deactivate users, admins need to remove them from each group they’re a member of — you’ll need to process all of these requests on your end.

All these inconsistencies are disastrous for security and compliance. If a user remains in a group they’re not supposed to be in, you risk exposing protected resources to unauthorized users or infringing on compliance regulations and internal policies.

Worst of all, the varying implementations from each IdP mean there’s no standardized protocol for resolving issues, making it challenging to troubleshoot when things go wrong.

Another example is push groups that only some IdPs like Okta support. Group Push lets you push Okta groups and their members to connected applications; however, it doesn't create groups in Okta. Users have to be assigned twice. Note that Okta advises using separate groups for push groups and group assignments. Otherwise, memberships won't be accurately reflected without manual group pushes for membership changes.

Let’s see a final example of how some IdPs handle group memberships differently. Instead of individually assigning users to a SCIM application, some IdPs, like JumpCloud and CyberArk, require that users be assigned to the application through group membership. To reflect valid user membership in your application, users should be removed from a group while the group is connected to the SCIM application rather than removing them directly from the application. Otherwise, you might still see users that are a part of that disconnected group. The same restriction applies to OneLogin for any user, which is only provisioned through group membership.

Onboarding fragmentation

Differences across IdPs means you need custom documentation and flows for each during onboarding.

Onboarding is the process of integrating your app with the customer’s identity provider (e.g. Okta). In order for your customer to do that successfully, you need to provide some form of guidance, be it documentation or an in-app walkthrough. Besides the initial connection, you need to map the existing roles and groups in the IdP to the roles and permissions that you have in your app. For example, if your customer’s IdP has team leaders in the “admin” group, and your app has the concept of an “owner” that you want to map them to, you need to direct your users towards how to create these mappings in their IdPs.

But the real problem here is that every IdP handles this app onboarding process slightly differently. To start with, each IdP has its own UI so you need different guidance for each one. But this is just the top of the iceberg since the fragmentation goes on when it comes to mapping roles and groups:

• Getting a user’s correct email address from Entra ID (Azure) requires special logic depending on whether that user is cloud-managed or synchronized.
• OneLogin has several different ways to provision groups.
• JumpCloud has a set of default attribute mappings that you need to work around.
• And many more.

Building this onboarding experience is a huge undertaking. And the more IdPs you have to support, the worse it gets.

Identifying unique users

According to the SCIM spec, the externalId attribute should be the unique identifier for a user between the IdP and your application. Unfortunately, this is not a hard requirement. Some IdPs respect it, while others will use permutations of a username and ID, which creates even more fragmentation.

IdP implementations fragmentation

Many more differences between IdP implementations of SCIM result in fragmentation. This section shows some of them.

Supporting Google Workspace

Google, which is the third most popular IdP on the planet after Okta and EntraID (Azure), does not support SCIM publicly. The only way to get directory information is to pull the information to your application, which is the opposite of the SCIM spec (where IdPs push). So to support Google Workspace in addition to another IdP, you now need to worry about creating a worker architecture with regularly scheduled jobs to poll this data, alongside your existing infrastructure for IdPs that do support SCIM.

Bulk operations

The SCIM spec refers to bulk operations as an optional feature. As a result, some IdPs allow bulk operations, while others do not. For example, Okta and Entra ID (Azure) do not.

Filtering results

Filtering results vary widely per IdP. One application might support filtering up to 100 results at once, another may have a much higher limit. Okta does not support query filtering using meta.lastModified. The list goes on.

Timing differences

Every IdP (except Google Workspace) pushes changes to all connected apps when something changes in a user or a group. The timing, though, of when this push happens differs from IdP to IdP. Some examples:

• Okta syncs changes in real-time.
• Entra ID (Azure), by default, sync changes every 40 minutes. For use cases where more immediate provisioning or deprovisioning of users, groups, or group memberships is needed, Entra ID provides provisioning on demand.

The solution: outsource your troubles

In this article, we discussed some of the challenges you would face if you decided to implement a homegrown SCIM solution. Although the list is not exhaustive, it’s enough to give you a sense of how difficult such an endeavor would be. Navigating the complexities of SCIM protocols and inconsistent behaviors across providers can be a minefield for developers and IT admins. The ramifications of these inconsistencies are not limited to technical headaches; they can spiral into significant security, compliance, and financial risks.

‍Directory Sync by WorkOS solves this issue by providing a more unified and secure SCIM provisioning solution. You only have to integrate with the WorkOS API and get support for every major identity provider on the market. WorkOS will automatically handle each provider’s intricacies and quirks, using an ordered, real-time Events API to pass requests to your app and smooth over any security issues.

• Get started fast: With SDKs for every popular platform, and Slack-based support, you can implement SCIM in minutes rather than weeks.
• Events-based processing: While webhooks are also supported, WorkOS’ unique Events API means every SCIM request is processed in order, and in real-time. You’ll never miss a provisioning request again.
• Pricing that makes sense: Unlike competitors who price by monthly active users, WorkOS charges a flat rate for each company you onboard — whether they’re syncing 10 or 10,000 users with your app.

To learn more about the challenges SCIM poses, see the following:

• Build vs buy part I: complexities of building SSO and SCIM in-house
• Implementation challenges of a homegrown SCIM solution
• SCIM complexity explained: Tackle group fragmentation

‍