Before creating a stream to start sending near real-time log data to the destination of your choice, consider the types of stream configurations available in DataStream:
- DataStream-managed. The new stream configuration type available in DataStream. Create and manage your stream directly in the DataStream API. You can create a stream configuration on either an active or inactive property. Logs will start delivering once the property is activated, or if the property is already active when the stream is configured.
- Property Manager-dependent. This configuration requires creating a stream in DataStream and setting up the DataStream behavior in Property Manager before the stream starts streaming logs. Useful for logging only specific subsets of traffic.
- Hybrid. Streams logging data for multiple properties at the same time, some of them with and without the DataStream behavior configured in Property Manager.
Stay updatedDataStream is a solution in constant development. We’re aiming to phase out Property Manager integration to improve workflow time, the user interface experience, and stream automation.
The DataStream-managed stream configuration is set to become the default type available in DataStream. It will gradually include features from Property Manager to the application.
See the Release notes to stay up to date with the latest developments.
DS-managed and PM-dependent streams
The DataStream dashboard shows if the stream is DS Managed, PM Dependent or Hybrid under the Integration Type category along with other stream metrics, including the number of datasets, destination, and stream status.
DataStream-managed streams may take less time to set up and start streaming logs, as you can configure each stream in the DataStream application. They offer improved support for automating resource creation, such as when using Terraform. You can configure features including logging midgress traffic and the data sampling rate when creating the stream, instead of setting them up in Property Manager.
Property Manager-dependent streams require additional property configuration, and may take longer to set up and start streaming logs, but allow leveraging the rule and match logic. This is particularly useful if you want to log only specific traffic segments on a property, compared to logging all traffic in DS-managed streams. You should also use PM-dependent streams if you want to apply different sampling rates for different properties monitored in the same stream.
See the table below for the breakdown of differences between both integration types:
| DS-managed | PM-dependent |
|---|---|
| Streams upload logs without configuring the DataStream behavior in Property Manager, as long as the property is active. | Configure the DataStream behavior in Property Manager and activate the new property version before an active stream starts sending logs. |
| Configure the sampling rate of data to log for each stream in the DataStream interface or API. | Configure the DataStream behavior for each property you want to include in the stream in Property Manager to set the sampling rate. |
| Enable logging midgress traffic (for example, between Akamai edge servers) in the DataStream interface or API. | Enable midgress traffic in the DataStream behavior in Property Manager. For details, see Midgress traffic. |
| Manage each stream using the DataStream interface or API only. The stream configuration reflects its actual properties. | The DataStream behavior configuration overrides any stream-level settings you configure in the DataStream interface or API. |
| Create multiple DS-managed streams for one property with different sampling rates and midgress traffic settings. | All DS-managed streams for one property follow the sampling rate and midgress traffic settings configured in the property configuration. You can leverage rules and matches in Property Manager to change between streams depending on various conditions. See Multiple streams support for details |
Hybrid streams
In DataStream, each stream can collect and send log data for multiple properties with and without the DataStream behavior configured in Property Manager. The DataStream dashboard lists streams with multiple properties as Hybrid.
In Hybrid streams, the settings configured in the DataStream behavior for each property in Property Manager (including logging midgress traffic and the sampling rate) always override the stream configuration from the DataStream interface or API.
Migrate a stream to DS-managed
DataStream now lets you quickly launch a DS-managed stream from scratch, but also change an existing stream configuration from PM-dependent to DS-managed.
For DS-managed streams, Edit a delivery (CDN) stream to include settings such as logging Midgress traffic and the stream's Sampling rate, and delete the DataStream behavior from your property configuration in Property Manager.
You can change your stream to DS-managed if you want to minimize configuration time and automate data collection in your stream.
From now on, use only the DataStream application or the API to edit an existing stream, including changes to the sampling rate, logging midgress traffic, or adding new properties for the stream to monitor.