Daily VCS Layout API
Pay-as-you-go

As was mentioned in Core concepts, one of the goals of VCS is for the engine to be lightweight, easily embeddable, and with clear performance guarantees even when features are combined. We want your compositions to render reliably and without surprises, both on Daily's cloud rendering system, as well as within your client applications.

To this end, VCS doesn't include CSS. It's a rich and powerful standard, but a full CSS engine is not compatible with the aforementioned goals. Instead, VCS breaks down CSS's responsibilities into separate properties available on the built-in components.

VCS's alternate approach to CSS

1. Styling

In VCS, a "style" strictly means the properties applied when generating the content of an element. Examples of style attributes would be font size or fill color. Styles are applied with the style prop, which you can find documented for each built-in component.

See the <Text> component's style prop as an example.

2. Compositing

Compositing refers to transformations and blending properties applied at the last stage when rendering the scene. In VCS, it's accessible by applying the transform and blend props. You can read more about this topic in the reference for the <Box> component.

3. Layout

This is the topic of this section. Layout is applied using the layout prop to VCS components.

Background

Video layouts are quite different from text documents and interactive application UIs. The layout engines available in CSS were designed for the latter use cases. The VCS layout engine is designed purely for video and presents a functional rather than declarative API.

A fundamental difference in design is that video tends to fill the viewport (i.e. the screen or video content area) by default. Traditional HTML documents are laid out as a flow that expands vertically outside the viewport. This doesn't apply to videos and — indeed — having content flow outside the viewport is generally undesirable.

When designing graphics for video, designers tend to split the available viewport into areas like "lower third" (for title graphics overlays) or "sidebar". The VCS layout engine takes inspiration here and operates on the same principle of splitting and shrinking the available frame. Every layout starts with the full viewport as the frame, and the tree of layout functions will gradually modify the frame as they get called. To create a "lower third", you'd simply write a layout function that returns coordinates where Y is adjusted down by 2/3 and height is reduced to 1/3. Any components nested inside will then receive this as the "parent frame" which they can adjust further.

Functional vs. declarative

In CSS, everything is expressed by declaring properties which the engine interprets to produce the final layout. It's powerful and often easy to use, but creates a certain engine bloat. There are hundreds of separate CSS properties, some of them conflicting with each other, and many even come with their own specific mini-languages tucked into string values that must be parsed and executed.

It's not possible for a HTML/CSS author themselves to add new layout models to the engine. There's a fixed set of engines available like "flexbox" and "grid". Since any author can request them at any time, the rendering engine must include everything. A modern web engine is a multi-hundred-megabyte affair.

VCS instead uses function-based layout that lets you plug in the exact code you need. It keeps memory footprint down, and more complex layout models can be provided as regular JavaScript libraries loaded as needed. There is no need to wait for the entire system to be updated if your layout engine is missing that one setting!

The `layout` property

One way to think of VCS's layout property is that it's like CSS display, but you get to decide exactly what it means and which configuration parameters it takes.

The value of the layout property must be an Array that contains one or two values:

The layout function (a Function object). (Mandatory)
Params passed to the layout function (an Object). (Optional)

For example:

1<Box layout={[ lowerThird, {pad_gu: 0.5} ]}>
2

Here we're passing a layout function named lowerThird and giving it a layout params object containing a property named pad_gu. (We'll return to this shortly.)

If an element doesn't have the layout property, it will simply inherit the parent frame, which is the entire viewport by default.

Let's look at a possible implementation of the lowerThird layout function:

1function lowerThird(parentFrame, params) {
2  let { x, y, w, h } = parentFrame;
3
4  h = Math.round(h / 3);
5  y += parentFrame.h - h;
6
7  return { x, y, w, h };
8}
9

The return value of a layout function must be an object with the properties x, y, w, h. This is called a "frame", and it describes a rectangle within the viewport in absolute pixel coordinates.

The most important input argument is parentFrame which is similarly a frame rectangle. The purpose of your function is to transform this rectangle.

For simple layout functions like lowerThird above, you don't need to think about what the frame units are; it's just slicing down the available frame using a rule contained in the function.

However, for more complex rules, you need to know about the wider context of where the layout is being applied. Let's look at that soon, but first a design detour…

You may have noticed that our previous lowerThird implementation didn't actually make use of the pad_gu param we passed in. What is the significance of the "_gu" suffix here? It refers to the "grid unit", a standard way to express in device-independent measurements in VCS.

See our moving watermark tutorial for a complete example on how to use the layout property.

The grid unit

The grid unit is a designer-friendly, device-independent unit. The default grid size is 1/36 of the output's minimum dimension. In other words, 1 gu = 20px on a 720p stream (and 30px on a 1080p stream).

The more informal definition is that 1 gu is a good minimum text size for a video stream. It's small but still readable on a TV screen. By defining values in grid units, they're automatically aligned on the default grid, and it's easier to get eye-pleasing results where things align, margins are multiples of a common base size, and measurements are guaranteed to scale correctly to different output sizes.

We recommend specifying design measurements in grid units where possible. For example, the font size for a <Text> component can be passed as grid units using the fontSize_gu style property.

It's currently not possible to redefine the grid unit size yourself, but this may eventually be added to Daily's streaming/recording API.

The layout context object

Returning to the lowerThird example above, we can now understand that pad_gu is supposed to express a padding measured in grid units. Remember that the layout function must return absolute coordinates. It seems like we need to access the grid unit → pixels scale factor somehow.

There is a third argument passed to the layout function. It's an object called "layout context" and provides a pixelsPerGridUnit property just for this purpose. Let's access the pad_gu param value and scale it. We can now complete the lowerThird function:

1function lowerThird(parentFrame, params, layoutCtx) {
2  const pad = params.pad_gu * layoutCtx.pixelsPerGridUnit;
3  let { x, y, w, h } = parentFrame;
4
5  h = Math.round(h / 3);
6  y += parentFrame.h - h;
7
8  // apply padding
9  if (pad > 0) {
10    x += pad;
11    y += pad;
12    w -= 2 * pad;
13    h -= 2 * pad;
14  }
15
16  return { x, y, w, h };
17}
18

The layout context offers the following properties:

pixelsPerGridUnit: Number. We saw this used in the above example. It's the scale factor to convert grid units to viewport pixels.
viewport: Object. This is a frame rectangle, so it has the properties x, y, w, h. (x andy are typically zero, but you should confirm this, rather than assume it.) Accessing the viewport frame is useful if you want to size something based on the viewport size regardless of how deep you're inside the layout tree.
useIntrinsicSize: Function. This is a "layout hook", a function that lets you access data and state from the layout system. Calling this function will return the intrinsic size of the current element to which your layout function is applied. For example, if your function is attached to an <Image> component, useIntrinsicSize will return the size of the image.

The last item gives us a hint of how the VCS layout system will be extended to provide future advanced functionality. For example, we're planning to extend the layout context with new hooks that will give you access to child sizes. This means your layout function will be able to call a hook function to effectively say "process my children first, then come back to me with their final computed size and let me adjust my frame accordingly". Keep an eye on the Daily blog or the VCS public repo on Github to learn about new features in VCS layout!

Daily VCS Layout APIPay-as-you-go

Daily VCS Layout API
Pay-as-you-go