How the SIEM event delivery service works (legacy customers only)
The content on this page deals with a legacy feature of the Akamai Identity Cloud. If you are currently an Identity Cloud customer and are using SIEM event delivery, that feature is still supported. However, if you’re new to the Identity Cloud, SIEM event delivery is no longer available.
The Identity Cloud’s SIEM event service is built on top of Amazon S3 (Simple Storage Service) buckets and the Kinesis Data Firehose data delivery system. In brief, events are:
- Generated within the Akamai >>PR.
- Are gathered together based on application ID.
- Are delivered via the Kinesis Firehose to an Amazon S3 storage bucket.
But, like we said, that’s the process in brief. Here’s a more detailed look at how the SIEM Event Delivery service does its thing.
As events are generated and added to the event delivery pipeline, these events are inspected by event consumers, software code programmed to check each event for a specific application ID. For example, suppose a consumer is programmed to look for application ID htb8fuhxnf8e38jrzub3c7pfrr, and a new event appears, an event associated with application ID 23qpduatarjrzdx3eh2ndcx38z. What does the event consumer do? In this case, nothing; after all, the consumer is only interested in application ID htb8fuhxnf8e38jrzub3c7pfrr.
Now let’s suppose that the next event that appears in the pipeline is associated with application ID htb8fuhxnf8e38jrzub3c7pfrr (we know the ID because that’s the value assigned to the event’s captureApplicationId property). This event is something that the event consumer is interested in; as a result:
-
The event consumer makes sure that the event type is not on the application’s SIEM event block list. If it is on the block list, then the event ignored.
-
If the event isn’t on the block list, the event consumer makes a copy of the event (the original event will be stored in Akamai’s data warehouse).
-
The relevant event details are packaged into a JSON object designed specifically for the SIEM Event Delivery service.
For example:
{
"id": "793d27fa-1391-46d1-a335-61d6c1055d4a",
"message": {
"app_id": "htb8fuhxnf8e38jrzub3c7pfrr",
"client_id": "nmub5w3rru9k6rzupqaeb7bbwv6jn658",
"endpoint_uri": "http://documentation.akamai.com/widget/traditional_signin.jsonp",
"event_type": "legacy_traditional_signin",
"forward_headers": [
{
"name": "HTTP_X_FORWARDED_FOR",
"value": "192.168.1.1, 192.168.1.2, 192.168.1.3"
},
{
"name": "HTTP_X_FORWARDED_PROTO",
"value": "http"
},
{
"name": "HTTP_X_FORWARDED_PORT",
"value": "80"
}
],
"ip_address": "192.168.1.1",
"origin": "https://login.documentation.akamai.com/",
"user_agent": "Mozilla/5.0 (Android 8.1.0; Mobile; rv:68.0) Gecko/68.0 Firefox/68.0",
"user_uuid": "437920f3-85dd-4cb7-ba8c-7025faea1d2c"
},
"msts": 1566206726081,
"type:" "siem#legacy_traditional_signin"
}
Curious as to what those cryptic parameters (msts?) and values actually mean? The article SIEM event details should answer all your questions. (Or at least your questions about SIEM parameters.) We might also note that SIEM event messages do not contain any personally-identifiable information: a message might include a user’s Identity Cloud UUID, but an event message will never include a user’s name, email address, phone number, etc. In addition, while event messages might tell you which attributes have been changed (e.g., email), those messages will never include the before or after values for that attribute (e.g., a user changed his email address from karim.nafir@mail.com to karim.nafir.jr@mail.com.
The JSON-formatted event is placed into an application-specific event queue, a sort of SIEM event “waiting room.” This process repeats itself each time until one of the following happens:
- 5 minutes have elapsed since the last time events were delivered to the customer, or,
- The total file size of all the events in the queue exceeds 128 MB.
Keep in mind that the 5-minute time interval between event deliveries is not a service level agreement: SIEM event deliveries will usually take place every 5 minutes, but this is not guaranteed. However, we do have a goal of ensuring that 99% of all event notifications are delivered within 15 minutes after the time the event occurred.
When either of the preceding conditions are met, all the events currently in the queue are bundled into a .ZIP file, with the filename consisting of the Kinesis data stream followed by a timestamp indicating when the file was created. That file is then delivered to an Amazon S3 bucket. If delivery cannot take place for any reason (for example, because of a network failure), the system will automatically attempt delivery again, and again (and again) until the file is successfully delivered or until 24 hours have elapsed. (Data can only remain in a Firehose queue for 24 hours until it is automatically expired.) Based on Amazon service level agreements, it’s all-but-impossible for any SIEM event file to not be delivered.
So what happens after events are delivered to your S3 bucket? (Each application enabled for SIEM events has a corresponding S3 bucket.) At that point, it’s up to you. Organizations will be given download and delete permissions to the S3 bucket. (But not write permissions: you’re allowed to take data out of the S3 bucket, but you’re not allowed to add data to the S3 bucket.) That means you can download (and, as needed, delete) .ZIP files from the S3 bucket whenever it’s convenient for you.
And then what you do with the zipped files after you download them? Again, that’s up to you. Based on the name alone, however, one obvious suggestion would be to unzip the files and import them into a SIEM data aggregation tool.
Whatever you decide to do with the .ZIP files, just be sure that you do it within 30 days: each file has a time-to-live value of 30 days. After 30 days in the S3 bucket, a file is automatically deleted and cannot be restored.
Updated almost 2 years ago