Webhooks v3 – the technology that forwards near-real time notifications when specified events take place on your website or in your app – works extremely well; in fact, you can make a case that it sometimes works too well. For example, suppose you have a large website that has hundreds of thousands (or even millions) of registered users. With a website that large, you’d expect to have quite a few users creating new accounts each day. You might also expect to have a sizable percentage of your users making changes to their accounts: adding a new phone number, updating their home address, agreeing to a new marketing consent. In turn, that could mean tens of thousands of daily user profile updates and, depending on how your system has been configured, tens of thousands of daily Webhook notifications.

Whether receiving that many event notifications each day is a good thing or a bad thing is something each organization must decide for itself. However, there’s little doubt that there are organizations who believe, at least when it comes to webhook notifications, that you can have too much of a good thing.

Why would they feel that way? In some case this might be because the information received isn’t especially useful or actionable: does it really matter that 400 different users updated the middleName attribute this morning? If it does, that's great: you can track that. But if it doesn't ....

Likewise, many organizations have more than one capture application: at the very least, organizations typically have a development application and a production application. To Webhooks v3, an application is an application: if an entityUpdated event is generated on the development application it will be forwarded long with entityUpdated events generated on your production application. At that point, it's up to you to determine which events you want to keep, and how to get rid of the events you don't want to keep.

And if that isn't enough, organizations might also run into problems trying to process all that data. After all, each webhook notification:

• Must be received.
• Must have its payload decoded.
• Must (or at least should) have its signature verified.
• Must be copied into a database or other long-term storage location.

And that’s assuming that you even want to copy every event notification to a database. After all, if you do get a bunch of notifications about users changing their middle name (and only their middle name) you might decide to delete those notifications. But that’s going to require even more effort on your part: in addition to the preceding tasks, you now have to examine each notification and determine whether it should be saved or discarded.

The point is this: if you receive thousands of notifications each day, that adds up to a lot of processing power, network bandwidth, and other listener endpoint overhead. And organizations don't always have processing power, network bandwidth, and other listener endpoint overhead to spare.

Webhooks filtering: a solution to this information overload problem

To help alleviate these issues, the Identity Cloud has added a filtering capability to Webhooks v3. What does this new capability do for you? Well, as we just noted, up until now the burden has been on you (and your listener endpoint) to identify and dispose of unwanted events. For example, suppose you created a Webhook subscription that requested notification each time a user updated their user profile. If you did that then you got exactly what you asked for: a notification any time any user modified any attribute in their user profile. And yes, that would include things like updates for applications you didn’t care about or for user profile attributes you didn’t care about. But there wasn’t much you could do about that: the Identity Cloud sent you everything, and it was up to you to go through that mass of data, saving the notifications you wanted and tossing out the notifications you didn’t want.

With filtering, however, the Identity Cloud can take over much of that decision making for you. Suppose you only want to receive notifications from your production application (e.g., application htb8fuhxnf8e38jrzub3c7pfrr). In the “old days,” the Identity Cloud would send you all your event notifications, regardless of application ID, and leave it up to you to pick out the notices for the production application.

Now, however, you can apply a filter to your webhooks subscription; when you do that, the Identity Cloud can check the application ID for each event before it sends an event notification to your listener endpoint. Is the application ID equal to htb8fuhxnf8e38jrzub3c7pfrr? In that case, the notification is sent. Is the application ID something other than htb8fuhxnf8e38jrzub3c7pfrr? In that case, the event is ignored and no notification is ever sent. That means that you – and your listener endpoint – don’t have to deal with unwanted events.

📘

Before you ask, yes, you can do more than just filter on application ID. See What you can filter on for more information.

How much difference can filtering make?

Suppose we have a simple little website that, over the course of a day, generates an average of 6 eventUpdated events. If we don’t apply a filter to that subscription then all 6 of those events are forwarded to our listener endpoint:

By comparison, suppose we apply a filter that only looks for events with the application ID 123456780. In that case, only 2 events are forwarded to the listener endpoint:

Admittedly, there’s not much practical difference between getting 6 notifications a day or getting 2 notifications a day. But suppose we’re getting 300 notifications each day? In a case like that, you can start to see a bit of a difference:

And what if we're getting 1,800 entityUpdated notifications, and not every day, but every hour? That begins to make a much more noticeable difference:

And if we also subscribe to the entityCreated and entityDeleted events, and if we also – well, you get the idea. The value of a webhook filter depends on such things as the amount of traffic your website generates; how often your users modify their user profiles; the information that's important to you and the information that isn’t important to you; etc. But it should be obvious that filters can potentially make a huge difference in the amount of traffic sent to your listener endpoint. In turn, that can make a huge difference in the network bandwidth and the processing power available to that endpoint, and make an equally huge difference to analyze and make sense of the incoming data.

Further reading

This article doesn’t represent everything we have to say about webhooks filtering; far from it. If you’re interested in learning more about webhooks filtering and how it works, we recommend that you start with the article What you can filter on If you’d like to learn the basics behind creating your own webhooks filter (by using a subset of the JSON Schema draft 7 validation language) you might want to skip ahead to the article Create a webhook filter. And if you want some sample code that you can copy and use practically as-is, well, we have you covered there as well: see the article Webhooks v3 event filter examples for both sample JSON filter code, and sample Curl code for adding that filter to a webhook subscription.