video

What it does

video is the task plugin for video clips. It owns the contract every video provider implements: VideoRequest, VideoResult, VideoJobPoll and VideoHandler. It exposes a small typed facade, app.video.*, for one-off generation, cost estimates and provider discovery.

Providers that serve this task today:

How it works

Video providers run long jobs, so a handler has two forms. It has estimate, plus either execute (one call) or the async pair submit + poll, or both. A value is a valid handler when estimate is a function and either execute is a function or both submit and poll are.

The two paths use the forms differently:

In the facade loop, done returns the result without the state field, failed throws the poll's error as it is, and an abort during the wait rejects at once with the signal's reason.

estimate runs at plan time, before the runner resolves build-file references. So a handler's estimate takes an EstimateRequest: a VideoRequest whose image, endImage, refs and fromDraft may still be { $ref } or { $file }.

The plugin is stateless. No lifecycle hooks, no events.

Config

pluginConfigs.video.

import { createApp } from "@moku-labs/ai";

const app = createApp({
  pluginConfigs: { video: { defaultProvider: "fal", pollIntervalMs: 2000 } }
});

API

Mounted at app.video, reachable with ctx.require(videoPlugin).

An unknown provider, or a registered value that fails the shape check, throws:

[ai] No video provider named "<name>" is registered.
  Available: <comma list or "none">.

VideoRequest

VideoFile is { path: string; mimeType: string; hash: string }.

VideoResult

VideoJobPoll

{ state: "pending" }, { state: "done" } & VideoResult, or { state: "failed"; error: unknown }.

Example

The durable path is a build file. The runner journals the job id, so an aborted run resumes by polling:

version: 1
name: shots
items:
  - id: s01-frame
    task: image
    provider: fal
    input:
      prompt: "a courier at a patisserie at night"
      aspect: "9:16"
  - id: s01
    task: video
    provider: fal
    input:
      model: minimax-h3
      prompt: "slow push-in on the courier"
      image: { $ref: s01-frame }
      seconds: 5

Draft, then final

A draft is a cheap preview clip. A final re-renders that draft at full quality. Today only ark supports it. The draft item sets params.draft: true. The final item sets fromDraft: { $ref: <draft item> } and may omit prompt. The final renders at 1080p only. Draft ids expire after 7 days.

version: 1
name: draft
items:
  - id: s01.draft
    task: video
    provider: ark
    input: { model: dreamina-seedance-2-5-260628, prompt: "slow push-in" }
    params: { draft: true }
  - id: s01.final
    task: video
    provider: ark
    input: { model: dreamina-seedance-2-5-260628, fromDraft: { $ref: s01.draft }, resolution: 1080p }

The facade is the direct path. Nothing is journaled:

import { app } from "./app";

await app.start();

const frame = { path: "frames/s01.png", mimeType: "image/png", hash: "a".repeat(64) };

const { usd } = app.video.estimate({ model: "minimax-h3", prompt: "slow push-in", image: frame, seconds: 5 });
console.log(usd);

const clip = await app.video.generate({ model: "minimax-h3", prompt: "slow push-in", image: frame, seconds: 5 });
await Bun.write("clip.mp4", clip.video);

console.log(app.video.providers()); // ["fal", "apimodels", "ark"]
await app.stop();

A custom provider registers a VideoHandler under "video". The async pair is enough:

import { createPlugin, registryPlugin } from "@moku-labs/ai";
import type { Video } from "@moku-labs/ai";

declare function startJob(request: Video.VideoRequest): Promise<string>;
declare function checkJob(jobId: string): Promise<Video.VideoJobPoll>;

const handler: Video.VideoHandler = {
  estimate: request => ({ usd: (request.seconds ?? 5) * 0.05 }),
  submit: async request => ({ jobId: await startJob(request) }),
  poll: async jobId => checkJob(jobId)
};

export const acmeVideoPlugin = createPlugin("acmeVideo", {
  depends: [registryPlugin],
  onInit: ctx => {
    ctx.require(registryPlugin).register("video", "acme", handler);
  }
});

Events

Emits

None.

Listens to

None.

Depends on

• registry: resolve("video", provider) and providers("video").

Related

• asset: register a face before a video item uses it.
• runner: the durable path for the same handlers.
• fal, apimodels, ark: the providers.