Mux

The Mux Developer Hub

Welcome to the Mux developer hub. You'll find comprehensive guides and documentation to help you start working with Mux as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started

Storyboards

Trick Play operations like fast-forward, rewind, scrub, and seek are made more visual with Storyboards. Before jumping into details, watch trick play in action:

Each image, also called thumbnails or tiles, you see when hovering over the scrub bar/timeline on the video player, is part of a larger image called Storyboard. A storyboard is a collection of thumbnails or tiles, created from video frames selected at regular time intervals and are arranged in a grid layout. Below image is the storyboard for the video, Tears of Steel, the same video used to demo trick play above:

The storyboard is downloadable using the Image URL in either jpg or png format.

GET https://image.mux.com/Dk8pvMnvTeqDk9dy5nqmXz02MM4YtdElW/storyboard.jpg

There is no need to download or generate the image URL when using storyboards for trick play. You need the storyboard metadata file for enabling trick play operations on the video player. This storyboard metadata is used as a metadata text track. The storyboard image is referenced from the metadata.

Storyboard Metadata

The storyboard metadata provides the x-axis and y-axis coordinates of each tile in the storyboard image and the corresponding time range. The metadata is available in both WebVTT and JSON format.

WebVTT

Most popular video players use WebVTT file for describing individual tiles of the storyboard image. You can request the WebVTT file using the VTT URL.

GET https://image.mux.com/Dk8pvMnvTeqDk9dy5nqmXz02MM4YtdElW/storyboard.vtt
WEBVTT

00:00:00.000 --> 00:01:06.067
https://image.mux.com/Dk8pvMnvTeqDk9dy5nqmXz02MM4YtdElW/storyboard.jpg#xywh=0,0,256,160

00:01:06.067 --> 00:02:14.067
https://image.mux.com/Dk8pvMnvTeqDk9dy5nqmXz02MM4YtdElW/storyboard.jpg#xywh=256,0,256,160

00:02:14.067 --> 00:03:22.067
https://image.mux.com/Dk8pvMnvTeqDk9dy5nqmXz02MM4YtdElW/storyboard.jpg#xywh=512,0,256,160

00:03:22.067 --> 00:04:28.067
https://image.mux.com/Dk8pvMnvTeqDk9dy5nqmXz02MM4YtdElW/storyboard.jpg#xywh=768,0,256,160

00:04:28.067 --> 00:05:36.067
https://image.mux.com/Dk8pvMnvTeqDk9dy5nqmXz02MM4YtdElW/storyboard.jpg#xywh=1024,0,256,160

00:05:36.067 --> 00:06:44.067
https://image.mux.com/Dk8pvMnvTeqDk9dy5nqmXz02MM4YtdElW/storyboard.jpg#xywh=0,160,256,160

WebVTT Compatible Video Players

The table below shows the various video players supporting the WebVTT files for trick play. If your player isn't listed here, please reach out, and we'll help where we can!

📘

Trick Play on iOS, Android and other platforms

Trick Play using WebVTT file may be limited to HTML5 browser-based video players and may not be supported in Device specific SDKs including iOS and Android.

iOS, Android, and other device platforms use HLS iFrame Playlist. Generating HLS iFrame Playlist is on Mux's roadmap.

JSON

There are many other scenarios for using storyboards. For instance:

  • Quick way of previewing the entire video can save the video editor/reviewer's time without requiring a full video playback
  • Seeking to visually feature rich scenes is a good way for selecting a Cover or Poster image of the video
  • Ease of developing trick play like functionality in Chromeless Video players like hls.js.

Using WebVTT file for metadata can be burdensome to implement. Storyboard Metadata expressed in an easy to understand & widely supported format like JSON helps in taking advantage of storyboards in new ways. Mux provides the same storyboard metadata in JSON format.

GET https://image.mux.com/Dk8pvMnvTeqDk9dy5nqmXz02MM4YtdElW/storyboard.json
{
  "url": "https://image.mux.com/Dk8pvMnvTeqDk9dy5nqmXz02MM4YtdElW/storyboard.jpg",
  "tile_width": 256,
  "tile_height": 160,
  "duration": 6744.1,
  "tiles": [
    {
      "start": 0,
      "x": 0,
      "y": 0
    },
    {
      "start": 66.066667,
      "x": 256,
      "y": 0
    },
    {
      "start": 134.066667,
      "x": 512,
      "y": 0
    },
    {
      "start": 202.066667,
      "x": 768,
      "y": 0
    },
    {
      "start": 268.066667,
      "x": 1024,
      "y": 0
    },
    {
      "start": 336.066667,
      "x": 0,
      "y": 160
    }
  ]
}

FAQs

Are signed URLs supported?

Yes. All storyboard URLs should have JWT when using signed Playback IDs and Audience, aud, claim code value should be set to s. See Signed URLs for more information.

Are there Cross-Origin Resource Sharing (CORS) requirements?

The storyboards URLs use image.mux.com hostname and stream.mux.com hostname is used for video playback URL. Because the URLs use different hostnames, it is recommended to add crossorigin attribute to the <video> HTML tag for access.

Updated 13 days ago

Storyboards


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.