Scaling user provisioning with SCIM bulk operations and filtering

When your system manages thousands of users across multiple organizations, efficiency and consistency are everything. SCIM (System for Cross-domain Identity Management) simplifies provisioning by offering a standard REST-based API for creating, updating, and deleting users and groups.

However, once you start operating at enterprise scale, standard endpoints like /Users and /Groups are only the beginning. To handle high-volume synchronization smoothly, two advanced SCIM features, bulk operations and filtering, can drastically improve performance and reliability.

Why SCIM is critical for scalable identity management

SCIM was created to standardize how identity providers (IdPs) such as Okta, Azure AD, or Google Workspace interact with service providers (SPs) like SaaS apps. It provides predictable endpoints and data structures, so developers can implement user management once and integrate with multiple IdPs effortlessly.

A typical SCIM flow includes endpoints such as:

	
POST /Users        # Create a user
GET /Users/{id}    # Retrieve a user
PATCH /Users/{id}  # Update user attributes
DELETE /Users/{id} # Deactivate or remove a user
	

These requests work perfectly for smaller environments. But if you are provisioning hundreds or thousands of users at once, sending an individual HTTP request per record can overwhelm both the IdP and the service provider. That is where bulk operations and filtering make a difference.

Bulk operations: process many changes in one request

Bulk operations allow a SCIM client to submit multiple create, update, or delete operations in a single request to the /Bulk endpoint. Instead of sending 500 separate POST requests for new users, the client can send a single batch that the server processes sequentially or in parallel.

Here’s an example request:

	
POST /Bulk
Content-Type: application/scim+json

{
  "schemas": ["urn:ietf:params:scim:api:messages:2.0:BulkRequest"],
  "Operations": [
    {
      "method": "POST",
      "path": "/Users",
      "bulkId": "user001",
      "data": {
        "userName": "alex@example.com",
        "name": { "givenName": "Alex", "familyName": "Nguyen" },
        "active": true
      }
    },
    {
      "method": "PATCH",
      "path": "/Users/9d4f12ab",
      "data": {
        "Operations": [{
          "op": "replace",
          "path": "title",
          "value": "Engineering Manager"
        }]
      }
    }
  ]
}
	

The server then returns a BulkResponse with individual results for each operation:

	
{
    "schemas": ["urn:ietf:params:scim:api:messages:2.0:BulkResponse"],
    "Operations": [
        { "bulkId": "user001", "status": "201", "location": "/Users/abc123" },
        { "method": "PATCH", "status": "200", "location": "/Users/9d4f12ab" }
    ]
}
	

Bulk operations reduce API overhead, help manage rate limits, and allow efficient rollback handling. For example, if one operation fails, you can retry only that item without re-sending the entire dataset.

!!The SCIM spec refers to bulk operations as an optional feature. As a result, some IdPs allow bulk operations, while others do not. For example, Okta and Entra ID (Azure) do not.!!

Filtering: target exactly what you need

Filtering makes it possible to retrieve a specific subset of SCIM resources, which helps reduce payload size and improve sync performance. You can use filters in combination with GET requests to retrieve only the data you need.

Example:

	
GET /Users?filter=active eq true
	

This returns all users who are currently active. You can chain conditions for more complex queries:

	
GET /Users?filter=title co "Engineer" and active eq true
	

You can also filter based on metadata such as modification timestamps:

	
GET /Users?filter=meta.lastModified gt "2025-10-30T00:00:00Z"
	

This approach is ideal for incremental syncs. Instead of fetching all users each time, you can request only those modified since your last sync checkpoint, significantly reducing both processing time and network traffic.

!!Filtering results vary widely per IdP. One application might support filtering up to 100 results at once, another may have a much higher limit. Okta does not support query filtering using meta.lastModified. The list goes on.!!

Combining both for high-performance provisioning

In a real-world scenario, filtering and bulk operations are often used together.

1. The IdP queries all users modified since the last sync using filtering.
2. The service provider receives that filtered list and processes all updates using the bulk endpoint.

Here’s a simplified workflow:

	
# Step 1: Fetch updated users
GET /Users?filter=meta.lastModified gt "2025-11-01T00:00:00Z"
# Step 2: Send them in bulk to the service provider
POST /Bulk
	

This combination minimizes API calls, reduces latency, and ensures data consistency across systems. It also makes error handling and retries more predictable since the operations are grouped and traceable.

Implementation tips

• Use idempotent design: When processing bulk operations, make sure each change can be safely retried without causing duplicates.
• Respect server limits: Some SCIM servers impose maximum bulk sizes (for example, 100 operations per request). Implement chunking logic on the client side.
• Test partial failures: A single invalid operation should not block the rest of the operations. Always handle BulkResponse entries individually.‍
• Paginate and cache filters: For large datasets, combine filtering with pagination using startIndex and count query parameters.‍
• Validate SCIM schemas: Ensure your user and group schemas are consistent with RFC 7643 and any custom extensions supported by your IdP.

How to set up Directory Sync with WorkOS

WorkOS makes it straightforward to integrate SCIM-based directory synchronization without building your own SCIM server. Here’s how to get started:

1. Install the WorkOS SDK: Choose your preferred language or framework (Node.js, Python, Ruby, etc.) and initialize the WorkOS client with your API key.

	
import WorkOS from "@workos-inc/node";
const workos = new WorkOS("sk_example");
	

2. Configure webhooks or the Events API: Set up a webhook endpoint in your app or poll the Events API to listen for dsync.user.created, dsync.user.updated, and dsync.user.deleted events. These represent user lifecycle changes from connected directories.

3. Enable Directory Sync in the WorkOS Dashboard: In the Admin Portal, enable Directory Sync and allow enterprise admins to connect their IdPs, such as Okta or Azure AD.

4. Map and store users: When WorkOS sends a user event, map the payload to your internal user model and update your database accordingly. This ensures your app stays in sync automatically.

Once set up, WorkOS continuously handles synchronization between your customers’ directories and your app, no matter which IdP they use.

Follow the quickstart for more details.

Bringing it all together

SCIM’s advanced features like bulk operations and filtering offer valuable strategies for scaling identity management, even though their availability depends on the identity provider.

WorkOS Directory Sync abstracts this complexity entirely, giving you a single, unified integration for provisioning and deprovisioning users across all major IdPs. By relying on WorkOS, your team can support enterprise ready directory sync without maintaining protocol-specific logic or handling SCIM edge cases directly.

Whether or not a connected IdP supports optional SCIM features, WorkOS ensures that your user data remains accurate, up to date, and compliant, so you can focus on building great products instead of managing identity infrastructure.