Vidmoat

WHERE THE MUSIC PEAKS AND WHERE IT HITS. Energy curve, brightness and onset density over time; sections detected by self-similarity (intro / build / drop or chorus / breakdown / outro) with a relative energy score; the single highest-energy moment; and the biggest energy jump. Plus an IMPACT MAP: hits, whooshes, risers and sub-bass drops with peak dB and decay length. Land your reveal on peakEnergyMoment or biggestEnergyJump, and align a title pop to an IMPACT rather than to the nearest beat — a hit is what the ear actually marks. hasMusicalStructure is conservative and declines on dense mixes; when it declines you still get correct boundaries and energy numbers, just unlabelled sections. No model, no credits.

SEE the video cheaply: classical pixel-math analysis (no vision model, no AI credits) of a video src — shot/cut boundaries, per-shot brightness/saturation/dominant color/motion score/sharpness, face presence + bounding box (YuNet, a small pretrained model — check faceDetectionAvailable if you need to know whether detection actually ran), letterbox detection, and overall "is this static/talking-head" signal. Use this BEFORE editing footage you have not seen — to find where cuts already are, avoid trimming into a blurry shot, know if a shot has heavy motion (steady it or lean into it), or center a reframe/crop on a detected face. Cheap and fast — call it freely, it never costs credits. Cut times are FRAME-ACCURATE: each is the first frame of the new shot, so splitting there leaves no frame of the outgoing shot behind. Per shot, motionClass (static/gentle/active/frenetic) is the same measure analyze_motion classifies on, and isStatic means motionClass static; colours are decoded the way the preview and export show them (colourDecode says which matrix). It does NOT return a tempo grid: for bpm, beatOffset and beatTimes call analyze_audio_context on the music.

Find real dead air with an actual voice-activity model (Silero VAD — pretrained, no AI credits), not just an amplitude threshold — so quiet speech is never mistaken for silence. Returns speechSegments and deadAir ({start, end, confidence, reason}) gaps between them. Use this instead of analyze_audio_context's silenceRegions when the footage has soft-spoken sections a plain loudness threshold would misflag as pauses (or vice versa). Slightly slower than analyze_audio_context (real inference, not just math) — prefer that one for a quick loudness/clipping check.

FLATTEN clips into one real video file — a pre-comp. Two things need this. (1) Transforming or grading a GROUP as one unit: without a bake, a "camera shake" has to be keyframed onto every layer by hand and they drift apart. (2) Splitting a composite: a live split keeps each half sampling its own patch of the ORIGINAL footage as it travels, so parting halves visibly drain of colour — bake first and each half carries the picture it was cut from. Costs no export quota and is never watermarked; this is an editing step, not a delivery. Returns a jobId — poll get_bake_status, which performs the swap for you when it finishes.

Stop waiting on an AI video generation or living image and free its one-at-a-time slot so a new one can start. Omit jobId to cancel whatever video is currently running. The provider or worker may still finish the original request, and credits already spent are not returned.

Cancel a render job from render_project that is still PENDING or PROCESSING, for example an export made obsolete by a later edit. A queued job is removed; a running one stops within a few seconds and produces no file. It then reads CANCELLED in get_render_status and no longer counts against the concurrent-render limit. The monthly export it used is not given back (a failed export keeps its count too). A job that already finished is left alone and the reply says so.

Stop a post from going out. It stays on the calendar marked cancelled (update_scheduled_post with a new time puts it back). Refuses a post that is being posted right now. Cancelling a post that ALREADY went out does not remove it from the platform; the result says so, and you must tell the owner. Get postId from list_calendar. Needs the Creator Studio subscription.

Create a new empty video project and return its id. Pass `workspace` to file it in a folder as it is created — see list_workspaces.

Get a one-time upload link for a LOCAL file (video, audio or image) that you can reach from a shell but cannot pass to a tool: the user's footage in your workspace. Send the file with the curl command it returns (curl -T <file> <uploadUrl>); the response is JSON with the durable url to use as an addClip src, exactly like import_media. The link works for 30 minutes, for this account only, within the plan's upload size. In ChatGPT, pass the attachment to import_media as `file` instead.

Create a workspace (folder) for grouping projects. Use it when the user is starting a body of work — a client, a series, a campaign — not for every project. Returns its id, which create_project and move_project_to_workspace accept.

Apply timeline editing commands to a project (add/trim/split/style clips, captions, keyframes, effects, transitions, and more). Commands are `{ op, ...params }` objects — discover them with get_command_schema. Returns per-command results PLUS layout lint warnings (overlapping text, offscreen elements, unreadable sizes) — always fix lint warnings before rendering. Pass previewAt:[times] to get rendered frames back as images in this same response and SEE your edit immediately. Omit projectId to edit the most recent project; set dryRun=true to preview without saving. Pass a requestId you invent to make the batch idempotent — a retry with the same id after a timeout will not double-apply.

START HERE on any file you have not edited before. ONE call that answers what an editor answers in the first ten seconds of opening something: is this a talking head, is there music already, is it noisy, is it multi-shot, does it need reframing for vertical, how many seconds of dead air are removable — each as a fact WITH the evidence beside it, so you can disagree with a conclusion rather than taking it on faith, and null WITH a reason wherever it could not be measured. Then recommendedFirstOps: the first three operations, ordered by how much they change the edit, each naming the tool that performs it. Cheap (it composes the other analysers), no credits. It samples a window rather than the whole file and says so in coversWholeFile.

Bring a STILL to life cheaply with vidmoat-image-live-1 (pilot: only on accounts an admin enabled). It moves only what the still already shows, measured: a camera move with real depth parallax (push in, pull out, drift, orbit, pan), ambient motion inside one region with everything else frozen (water, waterfall, waves, clouds, smoke, steam, fire, leaves, wheat, grass, hair, flag, curtain), or a gentle object motion on a cut-out subject (float, bob, sway, tilt). 8 credits a second, 2 to 10 seconds (5 s = 40), video only, no audio. Where the generative tier (G, our own Wan 2.2 model on our GPU) is available, faces breathing or blinking, people moving, vehicles driving, turning objects, and ambient or object motion described in words go to it: realistic motion of THIS picture, 24 credits a second, 4 to 5 seconds, 16:9, 9:16 or 1:1, no loop, about 2 to 5 minutes (the GPU wakes per job); every clip is checked so that nothing new appears (a person, hand, animal or vehicle that is not in the still) and is refunded if it fails. Camera moves, and any request that sets region, motion, box, strength or direction, stay on the cheaper tiers. It REFUSES, before charging, anything appearing, pouring, breaking or transforming (generate_video with imageUrl), and without the generative tier also faces, bodies, vehicles and turning objects. The output is MEASURED before delivery: a clip that does not move, moves outside the region, drifts from the still or gains a subject ends FAILED and the credits are refunded automatically. Asynchronous: returns a jobId; poll get_generation_status. Call with quoteOnly:true first to see the tier, method and price.

Generate original music or a song with Google Lyria (Studio). Describe mood, instruments, tempo and structure; optionally specify instrumental only or desired vocals/lyrics. Never use speech generation to read a musical brief as a substitute. Returns an audio URL and measured duration when available; inspect the sound before claiming musical content. Add the URL as an audio clip and duck only under verified speech.

Generate spoken audio from text using Auto, Grok or OpenAI text-to-speech (Creator-plan feature, cheap in AI credits relative to generate_image/generate_video). Synchronous — returns a url immediately, usable as an audio clip src via edit_project addClip, or just to hand the user a voiceover/spoken line.

Generate a transparent AI sticker from a text prompt (premium feature, uses AI credits). Returns a url to use as an image clip src via edit_project addClip. Styles: cartoon, kawaii, retro, neon, 3d, minimal. Pass rigReady:true to get an ANIMATABLE character instead of a flat sticker — see that parameter.

Start generating a short AI video using Grok/xAI or Alibaba HappyHorse (paid Studio only; trials and fully waived promotions excluded;, very expensive in AI credits). From a text prompt, or — by passing imageUrl — BY ANIMATING A STILL THE PROJECT ALREADY HAS, which is how you move a real logo, product shot or frame rather than inventing a lookalike. Asynchronous: returns a jobId; poll get_generation_status until status is COMPLETED. Only one generation can run at a time per account.

Poll a bake_clips job. While it runs, returns status/progress. When it succeeds it SWAPS THE PROJECT for you: the baked clips are replaced by one video clip pointing at the rendered file, with the originals stashed so unbakeClip can restore them. Takes a few seconds to a minute depending on length.

The owner's stored Content Blueprint: their niche, audience, edge, content pillars with a target mix, posting cadence, what to do more and less of, and a month of ideas, with how confident it is and what it was based on. Returns blueprint: null when none has been built, and stale: true when their work has changed since it was written. Never builds one. Read-only. Needs the Creator Studio subscription.

The owner's Creator Studio challenges (post five days running, ten posts through Studio, a thousand views on one video, use every connected channel) with current progress, target and whether each is done. Progress is computed fresh from their posts every call. Days are counted in the timezone given; UTC when omitted or unrecognised (the result says when it fell back). Read-only. Needs the Creator Studio subscription.

Get the full timeline command schema — every operation edit_project accepts, with parameters. Call this once before your first edit_project call.

Get one approved community effect by id, including its full GLSL shaderSource — pass effectId from list_community_effects. Apply the result with edit_project: addEffect{ clipId, type:"custom-shader", shader:<shaderSource>, marketplaceEffectId:<id>, colorA:<colorA.default if present> }.

The numbers on the owner's Creator Studio Overview, exactly as their screen shows them: connected channels, finished exports, posts published and views over the last 90 days, the best channel, twelve weeks of posting rhythm, the weekly streak, the top three posts, the next scheduled posts and the blueprint's pillar mix. views is null (never 0) when no post could be measured. Use this for "how am I doing"; use get_post_analytics for per-post detail. Read-only. Needs the Creator Studio subscription.

Read the authenticated creator’s optional onboarding defaults, suggested editing briefs and usual output format. Read before planning; explicit current instructions and existing project settings override these defaults. This grants no permission to spend or edit. Disabled personalization returns no active defaults.

How many AI credits are left on this account, and what the expensive operations cost. Check this BEFORE starting work you might not be able to finish: a long render or a batch of generations that runs out halfway leaves a half-edited project and a spent balance. Returns remaining credits, the plan, when the allowance resets, and a costs map.

Poll the status of a generation job started with generate_video or generate_live_image. Returns status PENDING/PROCESSING/COMPLETED/FAILED and, once COMPLETED, outputUrl. A living image (kind "live") also returns what was measured to move (motion.measured) and the credits charged and refunded; FAILED there means the motion check rejected it and the credits came back.

How the owner's posts are actually doing: views, likes, comments, watch time. A metric that could not be read comes back as null with a reason, NEVER as zero — treat null as "nobody counted", not as "nobody watched". The totals say how many posts they cover, because most platforms have not approved our app yet and their posts are genuinely unmeasured.

Get a project: settings, tracks, and a per-clip summary (id, type, name, start, duration, trackIndex). Pass full=true for the complete editor document including keyframes, effects and text styles.

Check a render job: status (PENDING/PROCESSING/COMPLETED/FAILED/CANCELLED), progress 0-100, and the download URL when completed.

Vidmoat craft skills (the Agent Skills SKILL.md format). With no arguments: the editing manual (the loop that works, the mechanics that cost other agents whole runs such as clips being top-level and setColor being 0-200 absolute, plus pacing, titles, audio and colour). READ IT ONCE at the start of a session. With name: that skill's SKILL.md (craft for one kind of edit or technique, or a specialist brief) and the list of its files; with name and file: one of those files. list_skills returns the catalog of names and what each is for.

Get a professional craft brief from one of the Vidmoat skilled agents: a film editor, colourist, sound designer, motion designer, short-form strategist, retention editor, documentary storyteller, ad creative director, interview editor, accessibility lead, brand guardian, music-video editor, trailer editor or explainer producer. Call this BEFORE planning an edit whose goal falls in one of those domains ("make it cinematic", "cut this for TikTok", "the audio sounds amateur"). Each brief carries the order of operations, the actual numbers professionals work to (loudness targets, safe zones in pixels, hold times, frames per beat, contrast ratios), a DO-NOT list, and an acceptance test you can verify from a preview frame. The brief comes framed exactly as the in-app agent reads it (how to apply it, the editorial direction, and which deterministic verifiers check that craft). Pass projectId and goal to have the specialist read the measured evidence of that project and return a cited beat sheet. Omit `id` to list every specialist with what it is for.

Import a user-provided vector Lottie JSON, GIF or animated WebP as a timeline animated sticker. No AI generation credits. Lottie: animationData object; GIF/WebP: base64 file bytes. Limits: 5 MB file (1 MB Lottie), 6 seconds, 180 frames; compiled at up to 256 px/30 fps, 2 MB embedded output. No external assets, expressions, images or fonts inside Lottie. Requires explicit projectId and requestId. Use search_editor_catalogue kind:elements query:animated for free built-in artwork instead. Playback controls use edit_project setStickerPlayback.

Bring media into the Vidmoat media store: a public media URL (video, audio, or image), or `file`, a file the user attached in this conversation (their own footage, music or photos). To upload a file from a machine you can run commands on, call create_upload_link instead. Returns a durable url to pass as `src` to edit_project addClip, plus the probed duration in seconds. Use this to bring footage, music, or images into a project. Not for 3D models (.glb/.fbx): bring those in with upload_media, then render them with the anim3d plugin (import_model, render_scene) and add the resulting video.

Inject a large, self-contained HTML/CSS element into a Vidmoat project as a new clip — for designed graphics too complex for the standard shape/text commands (lower thirds, title cards, stat blocks, gradient backgrounds). HOW IT RENDERS: the element is wrapped in an SVG <foreignObject>, rasterized ONCE to a still bitmap, and that same bitmap is drawn on every frame. It is never a live DOM node, so it obeys the rules of SVG-as-image, not the rules of a page. ANIMATING IT: two CSS variables are set by the rasterizer on every frame, so an element CAN move without being baked as N duplicated clips. --vm-p is clip progress (0 on the first frame of the clip, 1 on its last) and --vm-t is clip-relative seconds. Both are unitless, so compose them with calc(): a trim-path graph reveal is stroke-dashoffset:calc((1 - var(--vm-p)) * 1000); a radial chart wipe or donut is conic-gradient(... calc(var(--vm-p) * 360deg) ...); a spinner is transform:rotate(calc(var(--vm-t) * 360deg)); a count-up is a row of digit strips translated by calc(var(--vm-p) * -100%). Purely opt-in: mention either variable and the element is re-rasterized per frame, ignore them and it keeps its single cached bitmap and costs exactly what it did before. Drive the SHAPE of the motion with a keyframed scale/opacity on the clip if you want easing — --vm-p is linear. DOES NOT WORK (fails silently, no error): CSS animations and transitions — @keyframes never advance, so the graphic renders permanently at its starting state and anything beginning at opacity:0 stays invisible for its whole duration; animate the CLIP with addKeyframe on opacity/scale/x/y instead. backdrop-filter (nothing to sample — frosted glass comes out as a flat translucent panel; bake the tint into the background instead). mix-blend-mode (set the clip's blend mode with updateClip instead). position:fixed (falls back to static and the element reflows out of place — use position:absolute inside a position:relative parent). Web fonts, @font-face, and remote <img>/url() (no network during rasterization — use system font stacks and inline SVG or data: URIs). <script>, event handlers, and iframe/link/@import are stripped outright. WORKS: flexbox and grid, linear/radial/conic gradients, border-radius, box-shadow, text-shadow, filter (blur, drop-shadow), transform, clip-path, -webkit-background-clip:text, CSS custom properties, system fonts, inline SVG. EMOJI: supported, but each machine supplies its own emoji font (Segoe UI Emoji on Windows, Apple Color Emoji on macOS, Noto Color Emoji on the render server) — so the exported video will NOT match what the preview showed. They are also glossy shaded pictographs, which clash beside flat 2D vector shapes and single-weight icons; do not mix the two registers in one graphic. Prefer inline <svg> for iconography — it renders identically everywhere and inherits the design's stroke weight and palette — and reserve emoji for when a colour pictograph is genuinely the subject. Every one of these is checked after the edit and reported back in lint[].

What is scheduled to post, and what already went out. Returns each entry with its time, timezone, channels, status and — for anything published — the live URLs. Use this before scheduling so you do not double-book a slot the owner already filled.

List approved GLSL shader effects submitted by creators on fx.vidmoat.com (id, name, description, category, usage count), ordered most-used first. These render through the same sandboxed custom-shader pipeline as any hand-written effect — apply one with get_community_effect + edit_project addEffect. Applying one credits its creator toward a usage-based payout.

List this account's AI video generations and living images, running first (each with kind "video" or "live"). Use when generate_video or generate_live_image says one is already running: it returns the jobId to poll with get_generation_status, or to free with cancel_generation.

List the 20 most recently edited Vidmoat video projects (id, name, updatedAt, workspace). ALSO RETURNS `total`, `returned` and `truncated`: when truncated is true the list is a WINDOW, not the whole account, so never answer a "how many" question from its length. Pass `workspace` (a workspace name or id from list_workspaces) to list only what is filed there — on a busy account that is far more useful than the 20 most recent of everything, and it makes `total` meaningful again because it counts that folder only.

Browse published Recipes: title, what each makes, the techniques it uses and how many videos have been built from it. Returns previews only — building from one needs a recipe key, which the owner gets from the Recipes page.

The catalog of Vidmoat craft skills: each skill's name, what it covers and when to use it, its category (specialist, craft, reference, manual), version and files. Load one with get_skill and its name before planning the part of an edit it covers. Read-only.