Guide

Phase 2: User Acceptance Tests

Suggest Edits

After you are done with development tests, perform the following User Acceptance Tests (UATs).

Core validation tests checklist

1. Validate playback of on‑demand HLS and DASH assets

  1. Verify VOD organic playback is completed.
  2. Verify VOD seeks forward and backward.
  3. Verify VOD can restart after the media completes.

Expected results:

  • Events ready and playing fire within an acceptable time.
  • No fatal errors, nor unhandled player crashes.
  • Video renders correctly (no decoding artifacts), and audio is synced.
  • Accurate frame/resume and no stalls.
  • The playbackfinished event fires at completion.
  • Correct content duration and time position after seek.
  • Events seek and seeked are sent when seeking forward and backward.
  • Events play and playing are sent after the video restarts.

2. Validate live streaming behavior and DVR

  1. Test Live HLS.
  2. Test Live Dash.
  3. Test DVR windows seeking.
  4. Test Live Play/Pause (optional, for special requirements).

Expected results:

  • Playback is continuous.
  • Latency is within expected SLA.
  • Seeking in the DVR window returns expected content/time.
  • The timechanged event is triggered regularly during live streaming.

3. Validate adaptive bitrate and quality selection HLS and Dash

  1. Start with full bandwidth, then throttle to force a downswitch. Restore bandwidth to upswitch within reasonable time.
  2. Verify quality start level with startupBitrate property or maxStartupBitrate ( for Web SDK).
  3. Validate manual quality selection of bitrate from player controls.

Expected results:

  • Quality Change events triggered: Videoqualitychanged.
  • Playback continues smoothly.
  • Video and audio continue in sync.
  • Quality (resolution/bitrate) decreases when bandwidth drops and increases when restored.
  • No repeated oscillation between levels.
  • No severe rebuffering beyond defined thresholds.

4. Validate network auto-recovery and error handling

  1. Start live video playback and simulate network outage (3 to 10 seconds). Verify if the player retries and resumes playback automatically without page reload.
  2. Simulate a long outage and verify if the player emits appropriate error state.

For more information on network auto-recovery, see this Bitmovin community post.

Expected results:

  • The player retries per configured retry policy.
  • When connection returns, playback resumes without page reload.
  • Error codes are meaningful and logged.

5. Validate autoplay and muted behaviors conform to product requirements

  1. Verify that muted: true and autoplay: true start video playback in modern browsers.
  2. Verify that muted: false and autoplay: true don't start video playback, unless allowed.
  3. Verify that muted: true and autoplay: false don't start video playback.
  4. Verify that muted: true and autoplay: false don't start the video playback.

Expected results:

  • Modern browsers block autoplay of unmuted content, unless allowed.
  • The event ready is triggered even when the playback is blocked due to unmuted autoplay policy.
  • Muted autoplay works in Chrome, Safari, Edge.
  • The player provides a big play button when the playback is blocked.
  • The playback can be started manually when the autoplay is blocked.
  • The user is able to mute or unmute the player by using the mute button in the control bar.

6. Validate closed captions/subtitles

  1. Verify the enable/disable option for side-loaded WebVTT captions for VOD HLS/Dash.
  2. Verify the enable/disable option for side-loaded TTML captions for VOD HLS/Dash.
  3. Verify the enable/disable option for embedded captions CEA-608/708 VOD HLS/Dash.
  4. Verify multiple language tracks and switch languages at runtime.
  5. Verify seeking while captions are active.
  6. Verify default start configuration for closed captions enabled/disabled.
  7. Verify changing appearance: style, font, color.
  8. Verify live video with captions.

For more information, see Subtitles & Captions Support.

Expected results:

  • Captions display accurately and stay in sync through seeks and ABR changes.
  • Accessibility: closed captions can be toggled from the keyboard.
  • Events for live closed captions are triggered: cueenter, cueexit,cueparsed.

7. Validate Low Latency (LL-HLS or LL-Dash) and Startup/Live Latency

  1. Confirm LL-HLS can play with enableLowLatency on/off (requires special stream to validate).
  2. Confirm LL-Dash can play with enableLowLatency ULL on/off (requires special stream to validate).

For more information, see Low Latency Support .

Ads validation tests checklist

The advertisement plugin validation testing is divided into two types of technologies:

  • Client side ad insertion (CSAI), for example:
    • Google IMA SDK
  • Server side ad insertion (SSAI), for example:

For more information, see Setting up Ads with the Web Player.

Test the following use cases across ad plugins and technology you use:

  • Ad playback events (pre-roll, mid-roll, post-roll).
  • Quartiles are sent for each ad pod: first quartile, mid quartile, third quartile.
  • Ad break started and completed for live streams.
  • Join live streaming during ad break playback.
  • Video and audio of media assets continue in sync after ad break completes.
  • CC settings are retained after ad break completes.
  • Validate the player status if the device is locked during ad playback.
  • The player session is completed after post-roll ends.
  • Player controls during ad playback should not interfere with the ad vs video asset.
    • Video is not started if the ad is paused.
    • No audio overlap.
  • Validate ad break with multiple ad pods.
  • Validate the "Learn More" functionality.
    • "More Info" webpage is presented.
    • Video playback is paused while the user is in the advertisement website.
    • Ad playback resumes when the user returns from the website.
  • Validate skippable ads, if applicable.
  • Validate single inline ads, if applicable.
  • Validate non-linear inline ad, if applicable.
  • Validate bumper ads around ad breaks, if applicable.
  • Validate vast timeout settings.

Freewheel test scenarios for mobile

  • [Freewheel] [Simulate 403 FW_ADS_URL] [Video Plays normally]
  • [Freewheel] [Simulate 404 FW_ADS_URL] [Video Plays normally]
  • [Freewheel] [Audio-Focus] [Do not overlap with Video Content]
  • [Freewheel] [Seek before midRoll cuePoint once played] [MidRoll is not triggered again]
  • [Freewheel] [Seek after midRoll cuePoint] [MidRoll is triggered]
  • [Freewheel] [Background in PreRoll] [Ad Resumes when foreground]
  • [Freewheel] [Background in MidRoll] [Ad Resumes when foreground]
  • [Freewheel] [Background in PostRoll] [Ad Resumes when foreground]
  • [Freewheel] [Lock Screen in PreRoll] [Ad Resumes when unlocked]
  • [Freewheel] [Lock Screen in MidRoll] [Ad Resumes when unlocked]
  • [Freewheel] [Lock Screen in PostRoll] [Ad Resumes when unlocked]
  • [Freewheel] [Tap Over PreRoll] [WebPage displayed - Ad Resumes]
  • [Freewheel] [Tap Over MidRoll] [WebPage displayed - Ad Resumes]
  • [Freewheel] [Tap Over PostRoll] [WebPage displayed - Ad Resumes]

Analytics validation tests checklist

Usually, each analytics provider has its own set of capabilities and a dedicated visualization dashboard.

The following tests are general test scenarios focused on validating video quality of experience (QoE) and user engagement (for example, startup time, rebuffering, playback failures).

When performing UAT for analytics modules, focus on validating these two main elements:

  • Accuracy: Are the events and metrics correct?
  • Completeness: Are all relevant events captured consistently?

Initialization and session tracking tests

  • Verify that the analytics session starts when playback begins (check playerReady, playing event).
  • Verify that unique session IDs are generated per playback attempt.
  • Verify that metadata (custom data fields, for example, customData1–5) is correctly attached to sessions.
  • Verify analytics is initialized with the correct license key and endpoint.
  • Confirm GDPR/consent integration, if required (analytics blocked until consent given).

Basic playback events tests

  • Play a VOD to completion. Ensure startup, play, and ended events are reported.
  • Play Live video for 10 minutes. Ensure startup and play events are reported.
  • Start playback and then abandon it quickly before the video starts. Ensure startup events are reported.
  • Start playback, pause, and then resume. Ensure pause and play events are reported.
  • Seek to different parts of the video. Ensure forward and backward seeks are logged with correct timestamps/durations.
  • Simulate video rebuffing. Verify that buffering events (stall, resume) logged with accurate duration.
  • Mute and unmute audio. Ensure audio status is reported, if applicable.

Closed captions and subtitles tests

  • Enable and disable closed captions. Verify enabling/disabling captions generates analytics events.
  • Track selection. Verify that switching subtitle languages triggers correct metadata updates.

Custom metadata and errors tests

  • Set custom metadata in the configuration. Verify that your custom metadata appears in the analytics backend.
  • Trigger a player error. For example: network disconnect, invalid manifest, DRM failure (if used). Ensure player errors are reported in the analytics module.

Ads playback scenarios tests

  • Ad start/complete: Verify that ad impressions, start, and completion events are tracked correctly.
  • Ad quartiles: Verify that 25%, 50%, 75%, and 100% milestones are logged.
  • Ad errors/skipped: Ensure skipped ads and errors are reported.
  • Ad clickthrough: Verify click tracking works, if applicable.
  • Include ad breaks with multiple ad pods and bumper ads around.
  • Include ad breaks with pre roll, mid roll, and post roll ads.

Data validation against backend

Compare captured analytics data in the Analytics Dashboard with the expected:

  • Number of sessions.
  • Startup time distribution.
  • Average bitrate.
  • Buffer ratio.
  • Ad impressions.

Other validation tests

Verify functionality of the listed features.

  • DRM content playback
    • Validate Widevine and Playready Support.
    • Validate Fairplay Support for Apple devices.
    • Validate special request headers requirements.
    • See DRM Support for more information.
  • Casting Devices
  • Web content accessibility guidelines
    • Keyboard navigation.
    • Voice over announcements for each button in player controls.
    • Voice over announcements for close captions settings.
    • For more information, see Bitmovin Player Accessibility.

Updated 20 days ago