API workflow

In DataStream 2, you can activate a stream upon creating or save the configuration to activate at any time. Both require getting details necessary to configure a stream, including product and group IDs, destination for streaming logs, or data set fields you want to monitor.

Get stream details

Before creating a new or updating an existing stream, use this workflow to get the required details:

1. Run the List delivery products and associated groups operation to determine the group and the product that you want to create a stream for.

2. Run the List properties by group to determine the properties within the selected product that you want to monitor in the stream. Note that running this request returns only active properties. To list a property, activate it on the production network in Property Manager first. See Activate a property.

3. Run the List connector types operation to determine the destination where you want the stream to deliver log files. Keep the destination details for configuration.

4. Run the List data set fields operation to determine the data set fields that you want the stream to log.

Create and activate a stream

If you want to activate your stream on making the POST or PUT request, use this basic workflow:

1. Run the Create a stream operation to create and configure a stream or the Edit a stream operation to update a stream. Make sure you set activateNow to true to activate the stream on request.

2. Enable the DataStream behavior in Property Manager to start collecting and streaming logs.

3. Optionally, set the sample rate in the DataStream behavior to control the percentage of data you want to collect and stream.

Create and save a stream configuration

If you want to save your stream configuration and activate it later, use this basic workflow:

1. Run the Create a stream operation to create and configure a stream or the Edit a stream t a stream operation to update a stream. Make sure you set activateNow to false.

2. When you decide the configuration is ready for activation, run the Activate a stream operation.

3. Enable the DataStream behavior in Property Manager to start collecting and streaming logs.

4. Optionally, set the sample rate in the DataStream behavior to control the percentage of data you want to collect and stream.

Next, you can configure these optional behaviors and criteria to have better control over your stream:

• Custom log fields. Enable to log any of these data set fields: User-Agent, Accept-Language, Cookie, Referer, X-Forwarded-For or Custom field. See Custom log field.

• Criteria. In each property that's part of the stream, you can configure the DataStream behavior in a custom rule to log requests matching your criteria. This lets you filter by hostnames, paths, IP addresses, or other criteria. See Criteria in the Property Manager API.