DEV Community

Cover image for When My Motion Prompt Fought the Depth Reference
Joey
Joey

Posted on

When My Motion Prompt Fought the Depth Reference

I wanted a generated race car to take the same corner as the car in my reference clip. The depth-guided version came close. Then I added more motion instructions, and the car turned the other way.

Give the track its own reference

Before generating the shot, I made a clean still of the circuit based on the original video. I wanted the track layout and the barriers around the corner to be easy to read.

A depth video gives the model a record of how the distances in a scene change over time. In a busy shot, the layout can still be hard to read from depth alone. A separate scene image gave MiniMax H3 a clearer view of the track it needed to build.

I used the original clip to make the depth video. Astorie can now do that conversion directly on the canvas. The car images showed what the car should look like. The circuit still showed where the action happened. The depth video guided how the shot moved through that space.

Compare the turn before rewriting the prompt

I kept the car and circuit references the same. With the depth video and no added motion instructions, the starting position, camera tracking, and turn were closer to the source. With the extra instructions, the car started in a different place and turned in a different direction.

The turn is the easiest difference to spot. I included all three versions in my original racing-shot comparison on Astorie, including the run guided by a motion prompt alone.

If I get another shot like this, I’ll check the scene image first. Can I see where the corner begins and where the car should go? Then I’ll run the depth reference on its own. That gives me a baseline for seeing what an extra motion instruction actually changes.

Top comments (0)