Stream logs to Splunk

DataStream 2 supports sending logs to Splunk. It is an interface that lets you search, monitor, and analyze your data.

Depending on your choice, DataStream 2 can upload either uncompressed or gzip-compressed log files.

Optionally, you can upload a client certificate to enable mTLS authentication to improve stream security and prevent data delivery failures. The custom header feature allows you to optionally choose the content type passed in the log file, and enter the name and value for the header that your destination accepts.

Before you begin

To use Splunk as a destination for your logs, you need to:

  • Set up an HTTP Event Collector instance (HEC) that matches the type of Splunk software you use. Next, create a token and enable it. See Set up and use HTTP Event Collector in Splunk Web.

  • Save the HEC token that you enabled, and the URL for your event connector. The URL structure depends on the type of your Splunk instance. See Send data to HTTP Event Collector in Splunk Cloud.

How to

  1. In Destination, select Splunk.

  2. In Display name, enter a human-readable description for the destination. The name can't be longer than 255 characters.

  3. In Endpoint, enter the HTTP Event Collector URL to a Splunk endpoint, where you want to send your logs in the <protocol>://<host>:<port>/<endpoint> format. The URL can't be longer than 1000 characters. Example:

    https://<splunk-host>:8088/services/collector/raw
    

    DataStream 2 supports only Splunk HEC URLs for raw events. Entering endpoint URLs ending with /collector or /collector/event will result in an error.

  4. In Event collector token, enter the HEC token you created and enabled in Splunk.

  5. If you want to send compressed gzip logs to this destination, check Send compressed data.

  6. Click Validate & Save to validate the connection to the destination and save the details you provided.

    As part of this validation process, the system uses the provided credentials to push a sample request to the provided endpoint to validate the write access. In case you chose the Structured log format, the sample data appears in the 0,access_validation format. For JSON logs, the data follows the {"access_validation":true} format. You can see the data only if the destination validates, and you can access the destination storage.

Additional options

  1. Optionally, click Additional options to add mTLS certificates for additional authentication. In Client certificate, enter the:
    • TLS hostname matching the Subject Alternative Names (SANs) present in the SSL certificate for the endpoint URL. If not provided, DataStream 2 fetches the hostname from the URL.
    • CA certificate that you want to use to verify the origin server's certificate. DataStream requires a CA certificate, if you provide a self-signed certificate or a certificate signed by an unknown authority. Enter the CA certificate in the PEM format for verification.
    • Client certificate in the PEM format that you want to use to authenticate requests to your destination. If you want to use mutual authentication, provide both the client certificate and the client key.
    • Client key you want to use to authenticate to the backend server in the PEM (non-encrypted PKCS8) format. If you want to use mutual authentication, provide both the client certificate and the client key.

📘

When enabling mTLS authentication for this destination, set requireClientCert to true in Splunk if you want the endpoint to require certificate authentication when receiving log data. See Configure indexers to use a signed SSL certificate in the Splunk documentation.

  1. Optionally, go to Custom header and provide the details of the custom header for the log file:

    • In Content type, set the content type to pass in the log file header. application/json is the only supported content type at this time.
    • If your destination accepts only requests with certain headers, enter the Custom header name and Custom header value. he custom header name can contain the alphanumeric, dash, and underscore characters.

    You can use this feature for Splunk indexer acknowledgements passed as the X-Splunk-Request-Channel header. See Channels and sending data in the Splunk documentation.

🚧

Forbidden custom header values

DataStream 2 does not support custom header user values containing:

  • Content-Type
  • Encoding
  • Authorization
  • Host
  • Akamai
  1. Click Validate & Save to validate the connection to the destination and save the details you provided.

Did this page help you?