I build CutVoke, an open-source (MIT) local video editor that Claude Code / Codex can drive over MCP. This post is about one feature and its limits.
A request that keeps coming up in editing threads this month is some version of:
"Adapting finished edits from 16:9 to 9:16, 1:1, 4:5, etc. Reframing shots and repositioning graphics for different aspect ratios."
And from a Dev.to post on cutting Shorts from long talks (kelvaro):
"most of the clock time is not creative. It is the same checklist"
"Reframe to 9:16" is on that checklist. The usual way to do it is to duplicate the project file, change the canvas, nudge everything, and export. Now you have two projects that drift apart the moment you fix a typo in one of them.
Versions live inside one project
In CutVoke a "production version" is another sequence in the same project, not another file.
-
sequence.duplicatecopies the complete editable sequence and can setwidth,height,fpsandreframein the same step. Internal object IDs are regenerated and references updated, nested sequences included. -
sequence.configurechanges the active sequence's real canvas, frame rate and framing. -
sequence.switchmoves between them.
Canvas sizes are even numbers from 2 to 8192, frame rates 1–240. Changing the frame rate keeps absolute time, so clip speed doesn't change. Duplicating, configuring and switching are all undoable, and the original 16:9 sequence stays as it was.
Three framing modes, and what they really do
reframe takes one of three values:
-
contain: keep the whole picture (letterbox/pillarbox). -
cover: fill the canvas and crop from the centre. -
stretch: the old default, kept for older projects.
Positions and pixel font sizes are scaled to the new canvas; caption font size, stroke and shadow are scaled too, and default caption size is still picked from the canvas.
The honest part: cover is a geometric centre crop. It is not face detection and not speaker tracking. If your speaker sits on the left third, a centre crop will cut them. For two people in frame, contain or a layout you build yourself on extra tracks is the safer choice. CutVoke has no ready-made "two boxes stacked vertically" template; that is manual track positioning.
Letting the agent do the boring part, and look at it
Over MCP, the flow an agent can follow is roughly:
-
sequence.duplicatewith 1080×1920 andreframe: cover(orcontain). -
preview_framesto get up to 6 real rendered frames, captions included, and check headroom, hands, faces and subtitles. - Adjust the supported transforms/keyframes where the crop is wrong.
- Leave the edit for a human to open and fix by hand on the same timeline if the agent's crop choice is off.
The agent doesn't get taste for free. What it gets is pixels to check before anyone exports.
Queue every version at once
With an output path prefix set, "queue all versions" submits one export job per sequence, with the sequence ID in the file suffix. Each job freezes that sequence's canvas, frame rate, framing and timeline when it is queued, so editing afterwards doesn't change a job that's already submitted. export.enqueue and POST /projects/{id}/exports both accept a sequenceId, so you don't have to switch the active sequence first. Agents poll the job with export_job and wait the returned pollAfterSeconds between checks.
Exports re-encode (libx264, or NVENC when available). This is not a lossless stream-copy tool.
What this doesn't solve
- No subject/face-tracking reframe (centre crop only).
- No automatic highlight or "best 30 seconds" picking.
- No layout templates for split-screen verticals.
- Desktop only (Windows/macOS/Linux).
If the versions problem is where your week goes, the docs for this part are in docs/user/PRODUCTION_VERSIONS_INTERCHANGE.md in the repo: https://github.com/frankShan001/CutVoke. Questions and "this broke on my footage" reports are welcome in the comments.
Top comments (8)
Good that you say plainly that cover is a centre crop. From the motion-graphics side, the harder part after a reframe is usually the text and graphics, not the footage. Scaling positions to the new canvas can still leave captions or lower thirds under the platform UI in 9:16, since the bottom edge and right side of Reels and Shorts are covered by buttons and the app's own captions. I would add a safe-area overlay for each ratio to the preview_frames check, and set caption position per ratio instead of a scaled copy of the 16:9 one. Planning it at the edit stage also helps: keep the key action in the middle third and leave headroom, so a plain centre crop survives for 9:16 and 1:1. For a speaker sitting off-centre, a keyframed manual crop is still the reliable fix.
Thanks, this is a useful correction to the post: the footage crop is the easy half, the text is the hard half.
What CutVoke has today, so it's clear where the gap is:
preview_framesreturns to the agent.So your suggestion (a per-ratio platform overlay the agent can see in
preview_frames) is a real missing piece, not something it does now. Agreed on the edit-stage advice too: keeping the action in the middle third is what makes a plain centre crop survive, and for an off-centre speaker a keyframed manual crop is still the dependable fix.Appreciate you laying out what's there and what isn't. One practical thing from the editing side: the overlay doesn't have to be exact to be useful. Even a rough mask over the bottom fifth and the right edge of a 9:16 frame would catch most caption problems before export. Platform UI changes every few months anyway, so I'd keep it as a swappable image layer instead of hard-coding the numbers.
Good point, and it fits how the editor already works without new code. A rough guide can just be a PNG: put it on its own top video track in the 9:16 sequence, and it shows up in the frames preview_frames returns, so the agent sees the covered bottom fifth and right edge. Hidden tracks are skipped at render, so you turn that track's visibility off before export (or leave the guide out of the export sequence). When the platform UI moves, you swap the image, not code.
The part I'd still want is for that to be a proper "guide" layer that never renders, so nobody can forget to hide it. Thanks for the practical angle.
That is how a guide layer works in After Effects: it shows in the viewer but never renders, which is why a lot of motion templates keep platform-UI masks on one without any export risk. A flag like that on a track would fix the forgetting problem. Until then, one cheap safeguard is to make the guide PNG a loud semi-transparent red with a label baked in, so if it ever does get exported it fails in the first few frames instead of slipping through.
That's a better safeguard than anything I suggested: if the guide ever leaks, a loud labelled red block is caught at the first frame check instead of at upload. The After Effects guide-layer behaviour is exactly the model I'd want for a track flag. Thanks, I'll note it as a feature idea.
Keeping versions as sequences avoids file drift, and the honest distinction between geometric cover cropping and semantic reframing is important. I’d add a per-sequence review checklist for safe zones, captions, and subject framing, plus a visual diff at a few representative timestamps. That makes the automated conversion auditable without pretending that a centre crop is creative judgment.
Agreed, and "auditable without pretending a centre crop is creative judgment" is a good way to put it. Part of that exists today: preview_frames returns real rendered PNGs (captions included) at up to 6 timestamps, and preview_compare renders before/after pairs at the same timestamps for a command without committing it. There is no built-in per-sequence checklist yet; for now it's a habit of picking the same few timestamps in each version and looking at safe zones, captions and the subject. A saved checklist per sequence would be a reasonable next step.