Destinations

For your data stream settings, you can use a custom HTTPS endpoint or a third-party object storage location. Currently available destinations include:

Amazon S3
Azure Storage
Datadog
Elasticsearch
Google Cloud Storage
HTTPS

Loggly
New Relic
Oracle Cloud Storage
Splunk
Sumo Logic
S3-compatible

Within the destination argument of your request body, specify the destinationType, for example, HTTPS, and include all other required arguments for your destination type. See each destination type for details of its configuration.

Amazon S3

Argument	Required	Description
`accessKey`	✔	The Amazon S3 account access key for authentication.
`bucket`	✔	The Amazon S3 bucket's name.
`destinationType`	✔	The destination type. Set it to `S3`.
`displayName`	✔	The destination's name.
`path`	✔	The path to the log storage folder within your Amazon S3 bucket.
`region`	✔	The region where the Amazon S3 bucket resides.
`secretAccessKey`	✔	The secret access key used to authenticate requests to the Amazon S3 account.

{
  "destination": {
    "destinationType": "S3",
    "path": "ds2-logs",
    "displayName": "my-s3-destination",
    "bucket": "datastream.example.com",
    "region": "ap-south-1",
    "accessKey": "1A2bc3dEF4ghiJKL5mnOP6q7Rst8uVWxy8Z",
    "secretAccessKey": "ABD1CEFGH2IJK3LMN4"
  }
}

Azure Storage

Argument	Required	Description
`accessKey`	✔	The Azure Storage account access key for authentication.
`accountName`	✔	The Azure Storage account.
`destinationType`	✔	The destination type. Set it to `AZURE`.
`displayName`	✔	The destination's name.
`containerName`	✔	The Azure Storage container name.
`path`	✔	The path to the log storage folder within the Azure Storage container.

{
  "destination": {
    "destinationType": "AZURE",
    "displayName": "my-azure-destination",
    "path": "logs/edgelogs/{%Y/%m/%d}",
    "accessKey": "A12c3d4+5eF6ghiJK/KMn7OPqR8STUVWxy9ZaBCdEF01gh2iIKLMnOpQ==",
    "accountName": "my-account",
    "containerName": "rawlogscontainer"
  }
}

Datadog

Argument	Required	Description
`authToken`	✔	Your Datadog account's API key.
`destinationType`	✔	The destination type. Set it to `DATADOG`.
`displayName`	✔	The destination's name.
`endpoint`	✔	The storage endpoint URL for the logs.
`tags`		The tags you can use to filter and group metrics in your Datadog account.
`compressLogs`		A boolean that sets gzip compression for log files sent to a destination. If not set, it's `false` by default.
`service`		The name of the application or service generating the log events in Datadog.
`source`		The name of the source or the technology of the logs in Datadog.

{
  "destination": {
    "destinationType": "DATADOG",
    "compressLogs": true,
    "displayName": "my-datadog-connector",
    "endpoint": "https://http-intake.logs.datadoghq.com/v1/input/",
    "authToken": "av123cdef4ghi5ijkm67nopq8rstuv",
    "service": "datastream",
    "tags": "env:sqa,user:jsmith",
    "source": "java"
  }
}

Elasticsearch

Argument	Required	Description
`destinationType`	✔	The destination type. Set it to `ELASTICSEARCH`.
`displayName`	✔	The destination's name.
`endpoint`	✔	The storage endpoint URL for the logs.
`userName`	✔	The `BASIC` user name for authentication.
`password`	✔	The `BASIC` password for authentication.
`indexName`	✔	The name of the index within an Elastic Cloud cluster where you want to store log files.
`tlsHostname`		The hostname that verifies the server's certificate and matches the Subject Alternative Names (SANs) in the certificate. If not provided, DataStream fetches the hostname from the endpoint URL.
`caCert`		The certification authority (CA) certificate used to verify the origin server's certificate. If the certificate is not signed by a well-known certification authority, provide the CA certificate in PEM format for verification.
`clientCert`		The digital certificate in the PEM format you want to use to authenticate requests to your destination. If you want to use mutual authentication, you need to provide both the client certificate and the client key in PEM format.
`clientKey`		The private key for back-end authentication in non-encrypted PKCS8 format. If you want to use mutual authentication, you need to provide both the client certificate and the client key.
`contentType`		The content type to pass in the log file header.
`customHeaderName`		A custom header name passed with the request to the destination.
`customHeaderValue`		The custom header's value passed with the request to the destination.

{
  "destination": {
    "destinationType": "ELASTICSEARCH",
    "displayName": "my-elasticsearch-destination",
    "compressLogs": true,
    "endpoint": "https://elastic_rawserver01.elastic.com:1234/_bulk/",
    "userName": "jsmith",
    "password": "p@ssw0rd",
    "indexName": "elastic-search-index",
    "tlsHostname": "elasticsearch.example.com",
    "caCert": "-----BEGIN CERTIFICATE-----\\nMIID <sample - removed for readability> .... nMweq-----END CERTIFICATE-----",
    "clientCert": "-----BEGIN CERTIFICATE-----\\nMIID <sample - removed for readability> .... nMweq-----END CERTIFICATE-----",
    "clientKey": "-----BEGIN PRIVATE KEY-----\\nMIID <sample - removed for readability> .... nMweq-----END PRIVATE KEY-----",
    "contentType": "application/json",
    "customHeaderName": "X-ELASTICSEARCH-TAG",
    "customHeaderValue": "bulk"
  }
}

Google Cloud Storage

Argument	Required	Description
`bucket`	✔	The name of the storage bucket you created in your Google Cloud account.
`destinationType`	✔	The destination type. Set it to `GCS`.
`displayName`	✔	The destination's name.
`privateKey`	✔	A JSON private key for a Google Cloud Storage account.
`projectId`	✔	A Google Cloud project ID.
`serviceAccountName`	✔	The name of the service account with the storage object create permission or storage object creator role.
`path`		The path to the log storage folder within your Google Cloud bucket.

{
  "destination": {
    "destinationType": "GCS",
    "displayName": "my-gcs-destination",
    "bucket": "datastream",
    "projectId": "ds2-connect",
    "path": "log/edgelogs/{%Y/%m/%d}",
    "serviceAccountName": "ak-DS2",
    "privateKey": "-----BEGIN PRIVATE KEY-----\\nMIID <sample - removed for readability> .... nMweq-----END PRIVATE KEY-----\\n"
  }
}

HTTPS

Argument	Required	Description
`authenticationType`	✔	Either `NONE` for no authentication or `BASIC` for username and password authentication.
`destinationType`	✔	The destination type. Set it to `HTTPS`.
`displayName`	✔	The destination's name.
`endpoint`	✔	The storage endpoint URL for the logs.
`contentType`		The content type to pass in the log file header.
`compressLogs`		A boolean that sets gzip compression for log files sent to a destination. If not set, it's `false` by default.
`customHeaderName`		A custom header name passed with the request to the destination.
`customHeaderValue`		The custom header's value passed with the request to the destination.
`password`		The `BASIC` password for authentication.
`userName`		The `BASIC` user name for authentication.
`tlsHostname`		The hostname that verifies the server's certificate and matches the Subject Alternative Names (SANs) in the certificate. If not provided, DataStream fetches the hostname from the endpoint URL.
`caCert`		The certification authority (CA) certificate used to verify the origin server's certificate. If the certificate is not signed by a well-known certification authority, enter the CA certificate in PEM format for verification.
`clientCert`		The digital certificate in the PEM format you want to use to authenticate requests to your destination. If you want to use mutual authentication, you need to provide both the client certificate and the client key in PEM format.
`clientKey`		The private key for back-end authentication in non-encrypted PKCS8 format. If you want to use mutual authentication, you need to provide both the client certificate and the client key.

{
  "destination": {
    "destinationType": "HTTPS",
    "displayName": "my-https-destination",
    "compressLogs": true,
    "authenticationType": "NONE",
    "endpoint": "https://example.com/post",
    "contentType": "application/json"
  }
}

{
  "destination": {
    "destinationType": "HTTPS",
    "compressLogs": true,
    "authenticationType": "BASIC",
    "displayName": "my-https-destination",
    "endpoint": "https://example.com/post",
    "userName": "jsmith",
    "password": "p@ssw0rd",
    "tlsHostname": "kpiqaproxy.eastus.cloudapp.azure.com",
    "caCert": "-----BEGIN CERTIFICATE-----\\nMIID <sample - removed for readability> .... nMweq-----END CERTIFICATE-----",
    "clientCert": "-----BEGIN CERTIFICATE-----\\nMIID <sample - removed for readability> .... nMweq-----END CERTIFICATE-----",
    "clientKey": "-----BEGIN PRIVATE KEY-----\\nMIID <sample - removed for readability> .... nMweq-----END PRIVATE KEY-----",
    "contentType": "application/json",
    "customHeaderName": "X-Request-Header",
    "customHeaderValue": "Header-Value"
  }
}

Loggly

Argument	Required	Description
`destinationType`	✔	The destination type. Set it to `LOGGLY`.
`displayName`	✔	The destination's name.
`endpoint`	✔	The storage endpoint URL for the logs.
`authToken`	✔	The HTTP code for your Loggly bulk endpoint.
`contentType`		The content type to pass in the log file header.
`tags`		Tags to segment and filter log events in Loggly.
`customHeaderName`		A custom header name passed with the request to the destination.
`customHeaderValue`		The custom header's value passed with the request to the destination.

{
  "destination": {
    "destinationType": "LOGGLY",
    "displayName": "my-loggly-destination",
    "endpoint": "https://loggly_rawserver01.loggly.com/bulk/",
    "authToken": "ABCDE-12FG-GHI3-JKL4MN-OP56QRS",
    "tags": "raw,akamai_cdn",
    "contentType": "application/json",
    "customHeaderName": "X-Request-Header",
    "customHeaderValue": "Header-Value"
  }
}

New Relic

Argument	Required	Description
`destinationType`	✔	The destination type. Set it to `NEWRELIC`.
`displayName`	✔	The destination's name.
`endpoint`	✔	The storage endpoint URL for the logs.
`authToken`	✔	Your New Relic account's API key.
`contentType`		The content type to pass in the log file header.
`customHeaderName`		A custom header name passed with the request to the destination.
`customHeaderValue`		The custom header's value passed with the request to the destination.

{
  "destination": {
    "destinationType": "NEWRELIC",
    "displayName": "my-newrelic-destination",
    "endpoint": "https://log-api.newrelic.com/log/v1/",
    "authToken": "ABCDE-12FG-GHI3-JKL4MN-OP56QRS",
    "contentType": "application/json",
    "customHeaderName": "X-Request-Header",
    "customHeaderValue": "Header-Value"
  }
}

Oracle Cloud Storage

Argument	Required	Description
`accessKey`	✔	The Oracle Cloud account access key for authentication.
`bucket`	✔	The Oracle Cloud Storage bucket's name.
`destinationType`	✔	The destination type. Set it to `ORACLE_CLOUD_STORAGE`.
`displayName`	✔	The destination's name.
`namespace`	✔	The Oracle Cloud storage account's namespace.
`path`	✔	The path to the log storage folder within your Oracle Cloud Storage bucket.
`region`	✔	The region where the Oracle Cloud Storage bucket resides.
`secretAccessKey`	✔	The Oracle Cloud account secret access key for authentication.

{
  "destination": {
    "destinationType": "ORACLE_CLOUD_STORAGE",
    "displayName": "my-ocs-destination",
    "path": "my-path/{%d/%m/%Y}",
    "accessKey": "abcD1efgH2iJKl3mnoPqr45stuVW67890123xYZ",
    "region": "ap-hyderabad-1",
    "secretAccessKey": "abcD1efgH2iJKl3mnoPqr45stuVW67=",
    "bucket": "full-length",
    "namespace": "ab1cdefghi23"
  }
}

Splunk

Argument	Required	Description
`destinationType`	✔	The destination type. Set it to `SPLUNK`.
`displayName`	✔	The destination's name.
`eventCollectorToken`	✔	The Splunk account's event collector token.
`endpoint`	✔	The storage endpoint URL for the logs.
`clientKey`		The private key for back-end authentication in non-encrypted PKCS8 format. If you want to use mutual authentication, you need to provide both the client certificate and the client key.
`caCert`		The certification authority (CA) certificate used to verify the origin server's certificate. If the certificate is not signed by a well-known certification authority, enter the CA certificate in PEM format for verification.
`clientCert`		The digital certificate in the PEM format you want to use to authenticate requests to your destination. If you want to use mutual authentication, you need to provide both the client certificate and the client key in PEM format.
`customHeaderName`		A custom header name passed with the request to the destination.
`customHeaderValue`		The custom header's value passed with the request to the destination.
`tlsHostname`		The hostname that verifies the server's certificate and matches the Subject Alternative Names (SANs) in the certificate. If not provided, DataStream fetches the hostname from the endpoint URL.
`compressLogs`		A boolean that sets gzip compression for log files sent to a destination. If not set, it's `true` by default.

{
  "destination": {
    "destinationType": "SPLUNK",
    "compressLogs": true,
    "displayName": "my-splunk-destination",
    "endpoint": "https://http-inputs-customer.splunkcloud.com/services/collector/raw",
    "eventCollectorToken": "123-34a5-6b7c-890d-12ef-gh34",
    "tlsHostname": "kpiqaproxy.eastus.cloudapp.azure.com",
    "caCert": "-----BEGIN CERTIFICATE-----\\nMIID <sample - removed for readability> .... nMweq-----END CERTIFICATE-----",
    "clientCert": "-----BEGIN CERTIFICATE-----\\nMIID <sample - removed for readability> .... nMweq-----END CERTIFICATE-----",
    "clientKey": "-----BEGIN PRIVATE KEY-----\\nMIID <sample - removed for readability> .... nMweq-----END PRIVATE KEY-----",
    "customHeaderName": "X-Splunk-Request-Channel",
    "customHeaderValue": "ABCD1EFGH-23I4-56JK-789L-0123456MN789"
  }
}

Sumo Logic

Argument	Required	Description
`collectorCode`	✔	The Sumo Logic endpoint's HTTP collector code.
`destinationType`	✔	The destination type. Set it to `SUMO_LOGIC`.
`displayName`	✔	The destination's name.
`endpoint`	✔	The storage endpoint URL for the logs.
`contentType`		The content type to pass in the log file header.
`compressLogs`		A boolean that sets gzip compression for log files sent to a destination. If not set, it's `true` by default.
`customHeaderName`		A custom header name passed with the request to the destination.
`customHeaderValue`		The custom header's value passed with the request to the destination.

{
  "destination": {
    "destinationType": "SUMO_LOGIC",
    "compressLogs": true,
    "endpoint": "https://http-inputs-customer.splunkcloud.com/services/collector/raw",
    "displayName": "my-sumo-destination",
    "collectorCode": "abcd12EfGHIjKLMN4opPRstuV_WxyZ4-A56BC_DeF789GHIJK012LMN3==",
    "contentType": "application/json",
    "customHeaderName": "X-Sumo-Category",
    "customHeaderValue": "Header-Value"
  }
}

S3-compatible

Argument	Required	Description
`accessKey`	✔	The S3-compatible object storage bucket's account access key for authentication.
`bucket`	✔	The S3-compatible object storage bucket's name.
`destinationType`	✔	The destination type. Set it to `S3_COMPATIBLE`.
`displayName`	✔	The destination's name.
`endpoint`	✔	The storage endpoint URL for the logs.
`region`	✔	The region where the S3-compatible object storage bucket resides.
`secretAccessKey`	✔	The S3-compatible object storage bucket's secret access key for authentication.
`compressLogs`		A boolean that sets gzip compression for log files sent to a destination. If not set, it's `true` by default.
`path`		The path to the log storage folder within your S3-compatible object storage bucket.

{
  "destination": {
    "destinationType": "S3_COMPATIBLE",
    "displayName": "my-s3comp-destination",
    "accessKey": "1A2bC3D45EfghIJKL6mNopQ789RSTuVWzy0123ZAbCd",
    "secretAccessKey": "AB1CDE23FGHI456JKLMNO7",
    "bucket": "datastream.example.com",
    "region": "ap-south-1",
    "path": "ds2-logs{%Y/%M/%S}",
    "endpoint": "host.com"
  }
}

Updated 3 months ago