The most common AI video failure in my review folder is not a broken face. Not a sixth finger. It is everything moving at once. Hair, curtains, background crowd, camera, water — the model animates the whole frame because nobody told it what the shot is about. Motion brush and region control exist to make that one directorial decision physical. This moves. That holds still. Used well, they turn a busy, nervous clip into a shot with a subject.
The tool picture first — honestly — because this feature name gets abused in marketing. At the time of writing, Runway is the name most people associate with Motion Brush. It has shipped painted motion controls in its Gen-2 and later image-to-video workflows, with multi-region brushing in recent versions. Kling has also exposed motion-brush style controls in parts of its product and API surface. Other tools offer cousins of the same idea: masks, region selection, modify-region editing. Panels move around between releases, so treat any “click the third icon” tutorial as perishable. The method is what lasts. That is what this guide teaches.
The Problem: Everything Moves Or Nothing Does
Text prompts are a blunt instrument for motion. Write “a flag waves outside a café” and many models will animate the flag, the awning, the leaves, the waiter, and the whole camera. Write “still, calm, locked-off” and the same model often freezes the very thing you needed alive. So you end up arguing with the prompt in longer and longer sentences, while the frame keeps doing variety-show choreography behind your subject.
Storyboard-first planning helped me here before I ever touched a brush. On the board, each shot gets one main movement. One. A shot where the character turns, the camera pushes, the rain falls, and the sign flickers behind her is not four times better. It is four competing explanations for the viewer’s eye — and four things the model can get wrong. I write the main motion on the panel in plain words before generation: “hair and coat in wind, face still,” or “water moves, boat and fisherman hold.” If I cannot name the one movement, the shot is not ready to generate. Brush or no brush.
Region control is that board note made enforceable. Instead of hoping the model respects “background stays still” buried in a sentence, you show it the region. That difference — describing versus showing — is why a mask settles arguments that prompt rewriting never quite finishes.
How Region Motion Tools Think
Strip away the interface differences and nearly every motion-brush system asks for the same four decisions.
- The region. You paint or select the pixels allowed to move — or, in some tools, the pixels protected from moving.
- The direction. An arrow, a vector, or horizontal and vertical values that tell the region where to travel.
- The amount. Intensity or speed values that separate a drift from a thrash.
- The ambient spill. Some tools expose proximity or ambient motion — how much the painted motion is allowed to influence pixels nearby, like ripples spreading from a paddle.
Direction is where beginners — and I include past-me — overdo it. A painted arrow is a stage direction, not a command the physics engine must obey. Paint a curtain region with a hard sideways vector at high intensity, and you may get a curtain that slides like a garage door instead of cloth responding to air. So I start low. Small values first, review, then raise. Motion that is slightly too subtle can be sold in the edit with sound and a cut. Motion that is too big cannot be unsold by anything except a regeneration.
The other mental shift: painting a region does not detach it from the scene’s light and perspective. Brushing a boat to move across a lake still requires the reflection, wake, and scale change to make sense. Good region tools handle part of that automatically. None of them rescue a region painted across two different depth planes — say, half a person and the wall behind them — without a fight. Paint in depth layers: foreground, subject, background. The frame already thinks in layers. Your brush should too.
| Shot Need | What To Brush | What To Leave Alone |
|---|---|---|
| Product hero with living background | Steam, light shimmer, foliage behind the product | Product, label, hands holding it |
| Portrait with wind | Hair edge, loose clothing, background trees | Face, eyes, jewellery, collar line |
| Water or weather beat | Surface water, rain layer, smoke, flag | Architecture, signage, standing figures |
| Silent room, one small life cue | Curtain edge, plant leaf, clock pendulum | Everything else — restraint is the shot |
Brushing The Subject, Freezing The World
The classic use — and the right starting point if you have never touched the tool — is a subject that performs while the world holds its breath. Think of a character in a still room who lifts a cup. A product on a table that rotates a few degrees. A street performer framed against architecture that must not wobble.
My setup for this pattern is boring on purpose. Start from a still I have already approved — pulled from the storyboard panel, checked against the character or product sheet. Paint the subject region generously enough to include where the subject will be at the end of the move, not just where it starts. A hand that travels out of its painted zone mid-clip will smear at the border like it hit glass. Feather matters more than precision here. A slightly soft edge the model can blend beats a razor outline that cuts motion at the mask line.
Then I write a small prompt that agrees with the brush instead of competing with it. The brush says where. The prompt says what and how: “she lifts the cup to chest height and holds, natural pace, no camera movement.” Splitting the job that way stopped a specific recurring failure for me — the prompt saying “no camera movement” while the model still drifted the frame because ambient motion was left high. Text and brush both have to ask for the same stillness. If they disagree, the result is usually a compromise nobody wanted.
One warning from experience: brushed subject motion works best on short, single-idea moves. A lift. A turn of the head. A step forward. Ask a small painted region to perform a long, complex action — stand, cross the room, open a door — and the body mechanics fall apart long before the mask does. That is a storyboard problem again. Multiple boards, multiple shots. The brush directs a beat, not a scene.
Brushing The World, Freezing The Subject
The inverse pattern is the one I reach for more often now, especially in product and portrait work. The subject holds still — that is the job — and the world supplies the life. Steam off a coffee cup. Rain past a window behind a speaker. Trees moving behind a presenter. City lights breathing behind a bottle on a shelf.
Freezing a subject by painting everything except the subject sounds simple. Mostly it is. Two details decide whether it looks directed or just broken. First, contact points. If the subject’s hair, coat edge, or hands touch the moving world, some sympathetic motion is physically right — a coat hem that stays statue-still while wind screams past it looks pasted. I either include those edges lightly in the brush or accept a small amount of ambient spill around the subject. Second, reflections. A still product on a wet counter with animated rain behind it needs the reflection to agree, or at least not to argue loudly. When the reflection will not cooperate, I move the shot to the edit: generate the lively background beat separately and composite in Premiere Pro, where I control the blend frame by frame.
Where this pattern earns its keep commercially is text and labels. Animated backgrounds behind a frozen product keep the label readable for the entire shot. No morphing letters. No logo breathing. Anyone who has watched a generated label quietly rewrite itself mid-clip knows readability is not a minor detail. It is the difference between an ad and a screensaver.
What I skip, honestly: brushing busy crowd scenes into obedience. Painting nine background figures with nine motion zones is slow, fragile work, and the faces still misbehave at the edges. If the crowd is not the shot, empty the set on the board and add atmosphere with light, sound, and one background beat instead. A quiet, well-lit frame with one deliberate motion will out-perform a painted crowd in almost every review I have done.
When A Mask Beats A Prompt Rewrite
You do not need region control on every shot, and reaching for it reflexively slows you down. Plain image-to-video with a clean prompt handles plenty of shots in my pipeline — particularly in Seedance 2.5, where a well-built reference set already disciplines the frame. The brush earns its place at a few recognisable moments.
I reach for the mask after the second failed regeneration with the same complaint. That is my rule. First failure: reread the board, tighten the prompt, check the still frame — most failures live there, not in the tool. Second failure with the identical region misbehaving: stop writing sentences about that region and show the model the region instead. A third prompt rewrite against a problem the model has already misunderstood twice is credits burned on optimism.
Region tools are also the right call when stillness itself is the requirement — locked product, locked signage, locked face while weather moves — because “do not move” is precisely what prompts express worst. And they are the right call on insert shots: hands, objects, mechanisms, where the surrounding frame is set dressing that must hold still so the small action reads.
They are the wrong call when the underlying still is weak. This is the mistake I see most in other creators’ renders: brushing carefully over a starting frame that was never approved — wrong wardrobe, wobbly location, off-model face — then wondering why precise motion produced a precisely animated problem. The brush stages motion. It does not fix casting. That work happens earlier, on the board and in the reference sheets, where it is cheap. The reference-images guide covers how I keep those inputs honest before any motion tool gets involved.
Keep one discipline across all of it: review brushed shots at full speed first, then paused, then full speed again. Masks hide their seams at playback speed and confess them on pause — the soft tear where a painted region meets a still one, the halo around a frozen subject. If it only looks good paused, the audience will still feel the wrongness without naming it. If it only looks good in motion, it will survive a feed but die in a client review. I want both passes clean. Or I want to know exactly why I am accepting the flaw.
Frequently Asked Questions
Which Tools Actually Offer Motion Brush Controls Right Now?
Runway is the longest-running name here, with painted motion in its image-to-video workflow, and Kling has exposed motion-brush style controls in parts of its product. Interfaces and plan access change often, so check your current panel rather than an old tutorial — and learn the region method, which transfers between tools.
Should I Paint Tight Outlines Around My Subject?
No. Paint with margin for where the subject travels, and keep edges soft. Tight outlines shear motion at the mask border, which reads as smearing or tearing mid-clip.
Why Does My Frozen Background Still Drift Slightly?
Usually ambient or proximity spill is left up, or the prompt still asks for camera movement. Turn ambient influence down, state a locked camera in text as well as in the brush, and review — text and mask need to agree.
Here is the plain takeaway I give anyone learning this. Decide on the board which one thing moves. Enforce that decision with the brush only when prompting alone has argued back twice. And never use precise motion to rescue a frame you had not approved. Start with one portrait — hair moves, face holds — and you will understand more from that single shot than from a week of feature lists. Once a shot passes that test, it belongs in the Premiere timeline. Not back in the generator.