# Layers -- Agent API > Layers is a browser-based image/video editor. Its `window.LayersAgent` API > exposes UI operations as JSON commands. These include adding layers, painting > strokes, setting selections, and exporting images. This file is the contract > for software agents that control Layers. There are two ways to use it. ## 1. In-page (direct) Layers attaches `window.LayersAgent` after the app initializes. Every command is an async method that returns an envelope. ```js await window.LayersAgent.ready // resolves once bootstrap is done const v = window.LayersAgent.version // current API version (e.g. "1.0") const env = await window.LayersAgent.getState({}) // any command ``` Use this path from headless browsers, Playwright tests, or DevTools. ## 2. Via the layers-mcp sidecar (recommended for agents) `layers-mcp` is a Model Context Protocol server (stdio) that controls Layers in headless Chromium. Every LayersAgent command becomes an MCP tool. The server writes image and video exports to a configurable directory. It returns the file path in the envelope. - Repo: `https://github.com/noisefactorllc/layers-mcp` - Configure with `LAYERS_URL`, `LAYERS_MCP_OUTPUT_DIR`, `LAYERS_MCP_PROFILE_DIR` - Example client config: `examples/claude-code.json` in the repo Agents using an MCP client speak normal `tools/list` and `tools/call`. The underlying envelope shape is the same as the in-page API. The MCP server wraps the envelope in a content block. --- ## The envelope Every command response -- success or failure -- has this shape: ```js // success { ok: true, command: "getState", apiVersion: "1.0", result: { /* command-specific */ }, state: { /* full state snapshot */ }, warnings: [{ code: "...", message: "..." }] // optional } // failure { ok: false, command: "addLayer", apiVersion: "1.0", error: { code: "INVALID_ARGS_ENUM", message: "Unknown kind: ...", details: { field: "kind", got: "foo" } }, state: { /* full state snapshot */ } } ``` **Every response includes state, whether the command succeeds or fails.** Read `env.state` after each mutation. A separate `getState` call is unnecessary. --- ## Warnings `env.warnings`, when present on a success envelope, is an array of structured objects (not plain strings): ```js { code: "UNKNOWN_SETTING_KEY", // stable identifier; agents can switch on this key: "foo", // optional; the offending field if applicable message: "unknown setting key: foo (ignored)" // human-readable } ``` The response includes `warnings` only when the array is non-empty. Handle unknown warning codes. New codes may appear without an `apiVersion` change. Examples: - `UNKNOWN_SETTING_KEY` from `setSettings` -- the key was ignored - `THEME_PERSIST_FAILED` from `setSettings` -- localStorage write threw - `PROJECT_STORAGE_ERROR` from `listProjects` -- the storage layer threw an exception. The command returned the available result, usually an empty list. - `SESSION_TAKEN_OFFLINE` from `newProject`/`openProject` -- an online Seance session went offline first. These commands replace the whole composition, which a shared session cannot represent. --- ## State snapshot shape ``` state: { apiVersion, schemaVersion, project: { id, name, isDirty, canUndo, canRedo, canSaveAs }, canvas: { width, height }, view: { zoomMode, isPlaying, loopDuration }, foreground: { color }, selection: null | { kind, bounds, isEmpty, polygonPoints? }, layers: [{ id, name, sourceType, visible, opacity, blendMode, locked, transform, media, effect, drawing, children, mask }], selectedLayerIds, activeLayerId, jobs: [{ id, kind, status, startedAt, updatedAt, completedAt, progress, result, error }], // most recent 20 recentExports: [{ id, kind, filename, mimeType, sizeBytes, createdAt }], settings: { theme?, exportImage?, exportVideo? } } ``` `sourceType` is one of `effect | drawing | media | text`. `selection.kind` is `rectangle | oval | lasso | polygon | wand | color-range`. `jobs.status` is `queued | running | succeeded | failed | cancelled`. --- ## Tool discovery Don't hard-code command names. Discover them at runtime: ```js // In-page: enumerate live LayersAgent const commands = Object.keys(window.LayersAgent) .filter(k => typeof window.LayersAgent[k] === 'function' && !k.startsWith('_')) // Fetch the JSON-Schema-like map for each const { SCHEMAS } = await import('/js/agent/schemas.js') console.log(SCHEMAS.addLayer) // -> { type:'object', properties:{ kind:{type:'string', enum:[...]}, ... }, required:['kind'] } ``` `_`-prefixed commands (`_ping`, `_echoNumber`, `_echoEnum`, `_echoNested`, `_sleep`) are test/diagnostic only. Skip them. About 80 commands cover these operations: - Create, read, update, and delete layers. Set layer properties and transforms. - Set effect parameters and child effects. - Draw with `paintStroke`, `drawShape`, and `fillRegion`. Edit masks. - Create and modify rectangle, oval, polygon, lasso, wand, and color-range selections. - Manage projects, undo, and redo. - Control the view, foreground, settings, zoom, and playback. - Resize the canvas or image. Adjust levels, contrast, and white balance automatically. - Export images and videos. Install and list fonts. Manage jobs. --- ## Error codes | Code | When | |---|---| | `INVALID_ARGS_TYPE` | wrong type | | `INVALID_ARGS_REQUIRED` | missing required field | | `INVALID_ARGS_RANGE` | numeric out of range | | `INVALID_ARGS_ENUM` | not in allowed enum | | `NOT_FOUND_LAYER` | unknown layer id or child effect id | | `NOT_FOUND_STROKE` | unknown stroke id on a drawing layer (`eraseStroke`) | | `NOT_FOUND_EFFECT` | unknown effect id (response includes `didYouMean`) | | `NOT_FOUND_JOB` | unknown job id | | `CONFLICT_JOB_IN_PROGRESS` | a duplicate job is already running (e.g. installFontBundle) | | `JOB_LIMIT_EXCEEDED` | too many active jobs (cap = 20) | | `JOB_CANCELLED` | the job was cancelled (terminal state, not an error per se) | | `RESOURCE_DECODE_FAILED` | base64 / URL fetch / image decode failed | | `INTERNAL_ERROR` | unexpected handler throw | MCP-only: | Code | When | |---|---| | `JOB_TIMEOUT` | `waitForJob` timed out (sidecar default 120s for export, 600s for fonts) | | `JOB_NOT_SUCCEEDED` | job settled `failed` or `cancelled` | | `UNKNOWN_TOOL` | unrecognized MCP tool name | | `HANDLER_THREW` | sidecar handler threw outside the normal envelope | `details` always includes the field/value that caused the error. For `NOT_FOUND_EFFECT`, `details.didYouMean` is the 3 closest effect ids by Levenshtein distance. --- ## Jobs (long-running operations) Some commands return immediately with a `jobId` and run in the background: - `installFontBundle` -- downloads ~140 MB font bundle into IndexedDB - `exportVideo` -- frame-by-frame video encode (can take minutes for long clips) Pattern (in-page): ```js const { result: { jobId } } = await LayersAgent.exportVideo({ width: 1024, height: 1024, framerate: 30, duration: 5, format: 'mp4', quality: 'high' }) // Poll const job = (await LayersAgent.getJob({ jobId })).result // { id, kind, status, progress:{phase, current, total, message}, result, error } // Or block const settled = (await LayersAgent.waitForJob({ jobId, timeoutMs: 120000 })).result // settled.status === 'succeeded' | 'failed' | 'cancelled', plus settled.timedOut if applicable // Cancel await LayersAgent.cancelJob({ jobId }) ``` `JOB_KINDS` is exported from `/js/agent/jobs.js`: ```js import { JOB_KINDS } from '/js/agent/jobs.js' // JOB_KINDS.INSTALL_FONT_BUNDLE === 'install-font-bundle' // JOB_KINDS.EXPORT_VIDEO === 'export-video' ``` `installFontBundle` passes AbortSignal to the loader's fetch and extraction calls. Cancellation takes effect at chunk or font boundaries. A single font extraction cannot be interrupted. JSZip extracts a variable font of 5+ MB in one call. Cancellation during that call may take several seconds. `exportVideo` checks for cancellation at every frame boundary and before encoder finalization. **Via the MCP sidecar**, `tools/call` waits for `installFontBundle` and `exportVideo` to finish. The response envelope contains the completed job. Agents do not need to call `waitForJob` themselves. On timeout, the wrapper returns `ok:false` with `JOB_TIMEOUT`. For other unsuccessful job outcomes, it returns `ok:false` with `JOB_NOT_SUCCEEDED`. --- ## Exports & downloads `exportImage` and `exportVideo` both record an entry into `state.recentExports`: ```js { id, kind: 'image'|'video', filename, mimeType, sizeBytes, createdAt } ``` In the **in-page** path, exports trigger a browser download (``). The agent doesn't get bytes directly except via `exportImage`'s `bytes` field (base64 PNG/JPEG/WebP). Via the **MCP sidecar**, the Playwright runtime captures the download and writes it to `LAYERS_MCP_OUTPUT_DIR`. The envelope's `result.filePath` is the absolute path. For `exportVideo` it's nested as `result.result.filePath` because the envelope wraps the settled-job state. ### captureOnly + releaseExport `exportVideo({captureOnly: true})` skips the browser download. The completed job provides `result.result.blobUrl` and an `exportId`. The `blobUrl` remains valid until the page unloads or the agent calls `releaseExport({exportId})`. After reading each video's bytes, call `releaseExport({exportId})` to free its memory. Otherwise, the Blob remains in memory for the rest of the page's lifetime. In captureOnly mode, `exportImage` returns `result.bytes` as a base64-encoded image, plus an `exportId`. `exportVideo` returns `result.result.blobUrl`, an object URL that the caller can fetch. The envelope omits the raw Blob because envelopes must be JSON-serializable. Call `releaseExport({ exportId })` after reading the video to revoke its URL. ```js const env = await LayersAgent.exportVideo({ format: 'mp4', captureOnly: true, ... }) const settled = await LayersAgent.waitForJob({ jobId: env.result.jobId }) const { blobUrl, exportId } = settled.result.result const bytes = await (await fetch(blobUrl)).arrayBuffer() await LayersAgent.releaseExport({ exportId }) // frees the Blob ``` `exportImage`'s captureOnly path returns base64 bytes inline. There is no blob URL to release. Calling `releaseExport` with an image `exportId` throws `NOT_FOUND_EXPORT`. --- ## Patterns **Run a command and react to state:** ```js const env = await LayersAgent.addLayer({ kind: 'effect', effectId: 'NM/perlin' }) if (!env.ok) throw new Error(`${env.error.code}: ${env.error.message}`) const newLayerId = env.result.layerId // `env.state.layers` already includes the new layer; no extra getState call. ``` **Discover an effect:** ```js const env = await LayersAgent.searchEffects({ query: 'noise' }) // env.result is a list of { effectId, name, category, ... } const def = (await LayersAgent.getEffectDefinition({ effectId: 'NM/perlin' })).result // def includes param spec, defaults, name, category ``` **Make a rectangular selection and apply a fill:** ```js await LayersAgent.setRectangleSelection({ x: 50, y: 50, width: 200, height: 200 }) await LayersAgent.fillRegion({ layerId: '...', color: '#ff0000' }) ``` **Export and locate the file (via MCP sidecar):** ```js const env = await LayersAgent.exportImage({ format: 'png' }) console.log(env.result.filePath) // /Users/you/layers-mcp-exports/layers-2026-...-png ``` --- ## Caveats **Font registration is async.** If a layer's requested `font` is not registered in the browser, the first render uses Nunito as the fallback. Layers renders again after the font loads. The agent's snapshot reports the requested font immediately. The rendered pixels still use the fallback for at least one frame. To read output with the requested font, first call `installFontBundle` if the font is in the fontaine bundle. Alternatively, wait for one or two `getThumbnail` calls before reading bytes. --- ## Pointers - Source of truth: `/js/agent/index.js` (registration), `/js/agent/commands.js` (handlers), `/js/agent/schemas.js` (input schemas), `/js/agent/jobs.js` (job registry + `JOB_KINDS`), `/js/agent/snapshot.js` (state builder) - Test/diagnostic back door: `window.__LAYERS_TEST_HOOKS.jobs` (Playwright specs only -- NOT a public API) - MCP sidecar: `https://github.com/noisefactorllc/layers-mcp`. See its `README.md` for config and `examples/claude-code.json` for client wiring. - Commands return an MCP-content block when called via the sidecar. The inner JSON is the envelope described above.