AI Filmmaking: Hard Things Are Easy And Easy Things Are Hard
Working with AI to create movies can be very counterintuitive. Maddening, even. Some things are very easy to generate. And some things you’d think would take a long time take forever.
Take this scene: two characters sitting side by side in the bar. All I did was put in a simple prompt. Flow supports characters, so I gave it my two characters along with an image I’d generated earlier of the bar.
It came out well on the first generation. I generated two versions. It wasn’t perfect: Sadie is talking to herself, and it’s not exactly what I wanted, but it came pretty close, and it only took a few more iterations to get it the way I wanted, cuts and all.
Then there was the very first scene I set out to generate. All I wanted was Tom walking past the bar, quickly. It took forever. Eight generations.
It kept doing these weird things. The directions were clear: he doesn’t say anything, he just keeps walking. And of course the AI puts dialogue in anyway.
There seems to be this pull. The AI wants the character to do something, something narratively meaningful, some action, like talking. It took a while to rein that in, and it wasn’t easy. In one take the style was completely wrong and he just walks into the bar, when I’d told him to walk straight past. Another came a little closer. But why is he running? I told him to walk, not run.
What finally came out wasn’t exactly what I wanted either. Tom’s arms were up in a strange way, but I thought it added something, a little hook. He doesn’t walk the way you expect him to, so I kept it.
That’s why working with AI can be such a strange experience. I run into the same thing in my day-to-day work as a software engineer: things that used to take months can now take an hour, which is mind-boggling, and then something you’d think should take a minute takes half the day. It’s like it warps, and there’s an exhaustion factor there, and it can be a bit unnerving. I’m not complaining. I’m just saying that’s the way it is.
Check the video out here: