Starting a bulk export of failed events job

📘
Webhooks v3 bulk export of failed events is a new Identity Cloud feature currently in limited availability. Because the feature isn't in general release, that means that both the product, and this documentation, are subject to change at any time.

Starting a new job is as easy as using the POST method and calling the /exports operation:

curl -L -X POST 'https://v1.api.us.janrain.com/e0a70b4f-1eef-4856-bcdb-f050fee66aae/webhooks/subscriptions/e26925c7-ca17-4b82-8530-e43158c2d63a/events/exports' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer Yh1cEVzUOy0lZNaBxcGbQRQ6qFaRvW0UBqTD81p76ymSNN73P6eulMZFX2-MkN1h'

When you do this, Webhooks v3 first checks to see if you already have ajob in progress. If it does, your API call fails with a 409 Too many active exports error. That’s because a given region can only run one job at a time.

If no other jobs are running Webhooks v3 gathers up all your failed events and begins collating the file. When all the events are collected Webhooks v3:

Writes the file to an Amazon Web Services S3 bucket.
Creates a URL for downloading the file. This URL is listed as the self link in the operation response. The self link also appears when you query Webhooks v3 for information about your jobs.
Marks each event in the file as having been exported.

The latter point is important: events can only be exported once. For example, suppose you have events A, B, and C in your event store, and you start a job. Events A, B, and C will be included in the file. However, those events are also marked as having been exported. Suppose that, as soon as the job finishes, you immediately do another export. That export won’t include events A, B, and C. That’s because those events have already been exported.

Be sure to keep this in mind as well: files are only available for 7 days before they’re automatically deleted. We now have an file that contains events A, B, and C but suppose we wait 10 days before trying to download that file. By that time, we’ll be out of luck: the file will have been auto-deleted. In turn, that means that events A, B, and C are gone for good.

We should also note that, during its 7-day lifespan, you can download a file multiple times. Suppose you download a file but your computer dies before you can upload that file to your webhooks database. That’s fine: as long as 7 days haven’t elapsed you can simply download the file again.

And that’s a good question: suppose you start a job but there aren’t any failed events to export. In that case, Webhooks v3 writes a blank file to the S3 bucket and marks the export as a success. If you download an export file and the file size is 0 bytes that simply means that, when you started your export job , there weren’t any failed events to export.

When you start a job you’ll get a response similar to this:

{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "createdAt": "2019-08-24T14:15:22Z",
  "updatedAt": "2019-08-24T14:15:22Z",
  "state": "processing",
  "_links": {
    "self": {
      "href": "/e0a70b4f-1eef-4856-bcdb-f050fee66aae/webhooks/subscriptions/e26925c7-ca17-4b82-8530-e43158c2d63a/events/exports/497f6eca-6276-4993-bfeb-53cbbbba6f08"
    },
    "exports": {
      "href": "/e0a70b4f-1eef-4856-bcdb-f050fee66aae/webhooks/subscriptions/e26925c7-ca17-4b82-8530-e43158c2d63a/events/exports"
    }
  }
}}

You’ll also get back the HTTP status code 202 Export request accepted. Note that this the only status update you’ll get. To verify the current status of an export job, see Checking the status of a bulk export of failed events job.

See also