Last time I shared a tutorial on converting novels into videos. Today, I’m sharing how to create children’s picture books using Stable Diffusion, which is slightly easier than making videos.
A Little Idea
Most children or students now write essays. If we could convert their essays into a vivid little video, wouldn’t that enhance their sense of achievement in creation and stimulate their interest in reading and writing?
1
Preparation
I won’t introduce Stable Diffusion and Jianying here.
1. I previously shared this tool, Tiger ManCut, when discussing the conversion of novels to videos. If you’re interested, you can check it out. (This article only uses the intelligent storyboard and keyword reasoning features of this tool.)
Download link for Tiger ManCut: https://tiger.easyartx.com/landing
Usage and introduction of Tiger ManCut:
Stable Diffusion | Complete Process of Converting Novels to Videos
2. Install the Agent Scheduler plugin. Agent Scheduler is a batch image generation plugin within Stable Diffusion.
Github link: https://github.com/ArtVentureX/sd-webui-agent-scheduler
There are three installation methods:
1. Start Stable Diffusion, click “Extensions”—”Available to Download”—then click “Load Extension List”, search for the keyword “agent scheduler”, click “Install”, and restart Stable Diffusion after installation.
2. Install from the URL: Copy the Github URL and paste it into the “Git repository URL for the extension”, then click “Install” and restart Stable Diffusion after installation.
Plugin Github URL: https://github.com/ArtVentureX/sd-webui-agent-scheduler.git
3. Install the plugin package by downloading it from Github and placing it in the “extensions” folder within the Stable Diffusion directory.
2
Creating the Picture Book
Once all plugins and tools are installed, you can start creating the picture book.
1. First, launch Stable Diffusion and open Tiger ManCut.
2. Open Tiger ManCut. If you haven’t registered, you can click the top right to register. If you have registered and logged in, click “Create Video”, enter the “Project Name”, and then click “Create Project”.
3. Click the “Import Script File” in the middle, select the storyboard identifier as “Storyboard by Intelligent Parsing”, then click “Import File” to choose the prepared picture book. If you have already divided the scenes, it is recommended to import using “Excel”.
4. After importing the picture book, it will automatically perform intelligent storyboarding for us. After the storyboard is completed, click the “Infer Keywords” button in the upper left corner. Once the keyword inference is done, click “Next”.
5. When you reach the “Batch Drawing” stage, it will translate the keywords into English. Here, Tiger ManCut can be used, and we only need the storyboard and keywords. If you wish to continue using Tiger ManCut, you can click “Start Drawing” directly. This article will introduce using the Agent Scheduler plugin of Stable Diffusion for production.
6. Open Stable Diffusion that you just launched and choose a large model for creating the picture book.
For this large model, I chose manmaruMix_v30, which is very suitable for making children’s picture books. You can download this model from liblib and Civitai. (Thanks to the model author for sharing!)
7. Enter the positive and negative prompts. For the negative prompts, a general one will suffice. For the positive prompts, you can add some quality prompts at the beginning, followed by the keywords inferred from Tiger ManCut.
Oh, I almost forgot! Click the Agent Scheduler plugin in the menu bar, then click “Pause”. Before we finish adding all the scene prompts, it’s best to pause to prevent generation from starting with parameters not set correctly.
8. Set the parameters for generating the picture book based on your needs.
9. After setting the above parameters, click “Queue” under “Generate”. It will send the prompt words and parameters for the first scene to the Agent Scheduler plugin, and change the “Task ID” to 1 or the content of the first scene, making it clearer to see the parameters for each scene.
For the remaining scenes, just paste the prompt words for the remaining scenes into the positive prompt one by one, and click “Queue”. The generation parameters can remain unchanged.
10. After adding all the scenes, check to see if there are any omissions or parameters that need changing, then click “Resumer” in the upper left corner to start generating.
11. Once the images are generated, click “Task History” to view all the images generated. If you’re not satisfied with the results, you can double-click the list and adjust the prompts and parameters. After adjustments, click the regenerate button in the operation area on the right to regenerate the image for that scene. For satisfactory images, you can click save to save the images.
Once all images are ready, you can use an image processing tool to add text, thus presenting the static picture book. For those wishing to convert it into a video, you can continue reading.
3
Video Composition
Before making the video, we can generate an audio segment from the picture book content to facilitate subsequent subtitle generation.
1. Open TTSMaker, copy and paste the picture book content into the input box. Then, select the language and voice type. If you want to make more settings, you can click “More Settings”; if you want to keep it default, just click “Convert to Speech” to generate the audio.
TTSMaker link: https://ttsmaker.com/
2. After generation is complete, click to download the audio.
3. After the audio is generated, open Jianying and import the prepared images and audio into Jianying.
4. After importing, you can adjust the pauses in the audio. For example, if some pauses are too long, you can cut them shorter to make the listening experience more pleasant.
5. After processing the audio, you can start generating subtitles. Click the upper left “Text” – “Smart Subtitles” – “Transcript Matching”, paste the picture book content, and then click to start matching.
6. Once the subtitles are generated, select all subtitles and click the lock icon on the right to lock the subtitles. (Locking the subtitles makes it easier to align the images with the audio.)
7. This step is quite tedious, as you need to align the images with the audio. You can refer to the storyboard generated by Tiger ManCut while manually aligning the images with the audio (if you find it troublesome, you can complete this in Tiger ManCut).
8. Next, you can adjust the subtitle color and font slightly, and then add background music.
9. Adjust the background music to an appropriate length, set the volume of the background music to around -13.0dB, and the volume of the voiceover to around 8.0dB. Ensure that the background music does not exceed the volume of the voiceover.
10. Currently, the images are still static. You can make the images move slightly. For example, you can add keyframes for up and down or left and right movement for each image, making the entire video visually more dynamic and interesting.
I have scaled each image to 120% and added up and down movement keyframes (the starting frame for the first image is 100, ending frame is -200; the starting frame for the second image is -100, ending frame is 200, and so on).
10. At this point, the production is complete! If you want to add anything, you can add some sound effects, special effects, etc. After adjustments, click “File” – “Export” to export the video.
4
Conclusion
This concludes today’s sharing. Most children or students now possess writing skills. If we convert their essays into picture books or videos with vivid plots, or generate picture books featuring the children or themselves as protagonists, it will bring a unique sense of immersion, making the entire writing or reading process more enjoyable.
If this article has helped you or inspired you creatively, feel free to like, watch, and share, thank you~
Click below to follow our official account for more AIGC guidelines.
⬇️⬇️⬇️