Lesson 1 of 7
Movies from prompts
10 min read
You type a sentence — a paper boat drifting down a rain-filled gutter — and seconds later there's a short video of exactly that, moving, one that was never filmed. No camera, no crew, no editing. So where does a moving picture come from?
A clip out of thin air
Not long ago, a few seconds of video meant real work: set up a camera and shoot, hire a crew, or draw it frame by frame like an animator. And unless you filmed it, the scene simply didn't exist. Now you describe it in words and the tool makes a short moving clip — a little scene that was never in front of any lens. It feels like magic. It isn't: underneath is one surprisingly simple idea, repeated a lot, and the rest of this lesson takes it apart.
Describing a scene and getting a short moving clip is called text-to-video. You're not filming or editing existing footage — the tool makes brand-new frames from your words.
It starts as noise
Here's the surprise. The tool doesn't paint the clip stroke by stroke. It starts with a whole clip of pure noise — every frame just random speckle, like an untuned TV — and then cleans it up a little at a time, guided by your words, until a picture appears. And it doesn't finish one frame and move on: it cleans the whole run of frames at once, so they all sharpen into focus together. Drag the clean-up control below and watch a clip of noise resolve into a paper boat, every frame clearing in step.
Starting from noise and removing it step by step, guided by your words, is diffusion — the same idea image tools use, run here over a whole stack of frames at once.
Frames that agree
Cleaning up a stack of frames raises the real problem of video. Each frame has to be a good picture and agree with the ones around it — the same boat, the same colour, the same light — or the clip flickers and warps instead of flowing. So as it cleans, the tool also keeps the frames tied to each other, nudging every frame to match its neighbours. Turn that agreement down in the scene below and the boat stops being one boat: it jumps, changes size, slides off the water. Turn it up and the frames lock into a single moving thing.
How well each frame agrees with the ones around it is called temporal consistency. It's the hardest part of video — and it's what makes a run of frames read as one moving subject instead of a flickering slideshow.
Why the clips stay short
Two things fall out of this. First, cost: a single image is one frame, but a five-second clip is over a hundred frames, all cleaned up together and held in agreement — a huge amount of computing for a few seconds. Second, length: the further a clip runs, the harder it is to keep every frame agreeing, so small differences pile up and the subject slowly drifts. Both push the same way — today's clips are short, usually a handful of seconds, and you make longer videos by generating several and joining them end to end.
The tools that do this are improving fast — Google's Veo, Runway, Kling, Pika, Luma and others — and the field moves monthly, so treat any specific limit or price you read as a snapshot, not a fixed fact. You'll meet these ideas again as practical skills later in the course; for now, the mechanism is the thing to hold on to.
The gist
- —You describe a scene and the tool generates a short moving clip — it isn't filming or editing existing footage
- —It starts from a whole clip of noise and cleans it up step by step, every frame resolving together — that's diffusion, run over time
- —The frames must agree with each other or the clip flickers (temporal consistency); holding that up is hard, so clips stay short and cost a lot
A friend says making a five-second AI video is basically the same as making one image — you just click generate once. What's the better description?
Continue in the app
Take the whole AI Video course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.