Hello!
You’ve probably already seen a bunch of AI-generated images and videos. Have you ever wondered how they’re created? Today, I will share with you the entire process of how you can create a similar video using just two free programs and your laptop.
In this article, you will learn:
- What the results look like
- How to set up and run the programs
- How to control the behavior of the AI
- How Instagram users react to AI-generated content Let’s also conclude with a full video.
Results of Video Creation
We have two different results.
| Option 1 | Option 2 |
|---|---|
 |
 |
Option 1 involves feeding video frames to a stable diffusion with a static prompt, essentially acting as an advanced filter for the video.
Option 2 involves feeding the freshly generated AI image as input for a new one and modifying the prompt on the fly. As you can see, the AI can go quite wild, so the parameters should be carefully chosen.
Setup and run
Before we dive into settings and code, let’s look at what to install on our local machine. I use a Mac M1, but this should work for Windows as well.
Step 1. Install AUTOMATIC1111 software.
Follow the Readme instructions and then run it locally:
./webui.sh --share --skip-torch-cuda-test --no-half
Now you have your local version of the Mid journey. Play with the UI, and whenever you’re ready, continue with Step 2.
Step 2. Clone and run the custom script:
I built a simple NodeJS program (GitHub) that takes a video as input, along with parameters, and produces a sequence of frames modified with AI.
# get copy of a program
git clone https://github.com/skobak/video-to-ai
# open the folder
cd video-to-ai
# we need to have node >17
nvm install 17
nvm use 17
# install deps
npm i
# run programm with default settings
node main.js
Now, you can enjoy the process of creating your first AI video with automation!
Settings and Configurations
As always, the result comes with experimentation. For Stable Diffusion, two main parameters significantly affect the results:
- CFG Scale Affects how wild the AI will go with a prompt
- Denoising strength Affects how far it will deviate from the original image
You can play with these results by providing flags: -c and -d.
#example
node main.js -c=14.4 -d=0.5
Let’s look at how these properties affect the end result:
| Original | Slightly modified | Go wild |
|---|---|---|
 |
 |
 |
| c=0,d=0 | c=7.5,d=0.3 | c=25.4,d=0.6 |
Feel free to play with script parameters:
-v <video path>
-p <prompt>
-i <prompt input file path> !NB, this flag will run Scenario 2
-f <number of frame>
-c <CFG Scale, creativity scale 1-30>
-d <Denoising strength, 0-1>
-s <seed>
Example:
# This command will create an AI version of each frame of video.mp4
node main.js -p="Make it look like a painting in modern style" -v='./video.mp4'-c=14.4 -d=0.5
Or
Or, go to ChatGPT and use the following prompt:
Make a 25 prompts, one per line separated by comma in CSV format
that will generate the different style of painting of a cat in different artistic styles
Copy the result into prompts.txt file in a root of script folder and then run:
node main.js -i="./prompts.txt" -v='./video.mp4' -c=14.4 -d=0.5
Here we go, enjoy the result!
Look for reaction in Instagram
What about the product side of our experiment? We’ve been developing our cat’s Instagram account and uploaded normal and AI-generated videos as reels to find out how they perform:
I haven’t found a direct correlation — simple reels can go viral without any AI. However, AI can serve as a useful tool in your toolbox.
Conclude/Full Video
Today, we acquired knowledge on automating AI video creation, identified the tools necessary for the process, and learned how to construct a custom script. My main insight is that in the present era, we can swiftly develop extraordinary tools by merging AI with automation and scripting.
Next time, we will test and refactor our code using Co-Pilot and Chat GPT. Our goal is to discover methods of making our solution not only easier to maintain, but also thoroughly tested.
I hope you found this intriguing. Please feel free to provide feedback! I aim to delve deeper into AI tools, both here and on my YouTube channel.
Don’t forget to subscribe and share!
🐯 Cat account
Source code
My youtube
Best,
Alex
Top comments (0)