Learn AI Filmmaking Day 4 — Characters Who Act
Day 4 of 7: references, beats and dialogue
Series note. This is the fourth of seven articles that teach AI filmmaking from the ground up. So far our films have starred strangers, people the model invented on the spot and would never draw the same way twice. This article designs a character, teaches the model her face, and gives her two lines to say.
The film this article builds: 25 seconds from a single generation. Watch it with sound.
The brief
By the early 1930s Disney’s animators had a problem we would recognize. A dozen artists were drawing the same mouse, and every one of them drew him a little differently, so the studio began pinning model sheets above the desks: one character from the front, the side and the back, with notes on proportions and the things that must never change. The sheet did not make anyone a better artist; it made everyone draw the same character.
A video model has the same problem and the same fix. Every generation is a new artist with no memory, and unless you hand it the sheet, it invents a new person. The good news is that the current models take the sheet directly, as reference images, and hold a face across a whole shot.
So here is the brief. We are going to make a twenty-five-second film with one designed character, one supporting character, two acting beats and two short lines of dialogue, from a single generation with reference images. Its logline: an eight-year-old scientist in a lab coat runs a formal taste test of her lemonade on her grandfather, and the verdict is a Nobel Prize.
Two skills stack in this article: making a character the model can hold, and making her act. Figure 4.1 starts with the first, because a character the audience cannot read is not worth keeping consistent.
Figure 4.1: What makes a character readable on screen, before a single frame is generated.
A character you can read
Screenwriters say a character should be recognizable from the silhouette alone, and animators have proved it for a century. Before consistency, before performance, a character needs five things the audience can read in a glance: a silhouette, one distinguishing mark, wardrobe that tells the job, a movement style and a voice.
Our scientist is called Penny. Her silhouette is an oversized lab coat and a round puff of curly hair; her mark is a pair of clear safety goggles pushed up into that hair; her wardrobe says scientist and says kid at the same time (the coat over a yellow t-shirt, mismatched striped socks); her movement is precise and still; and her voice is small, flat and deadly serious. Her grandfather, Gus, gets the same treatment in fewer words: big, brown cardigan, reading glasses on a cord, a thick grey mustache, slow and warm.
Think of these five lines as the character header, the block of text the book calls a labelled character description. It goes at the top of every prompt Penny appears in, and it never changes. Remember, the model cannot keep consistent what you never defined.
The character sheet
The model sheet is where the design becomes something the model can hold. Figure 4.2 shows the anatomy: the same character three times, front, three-quarter and profile, full body from head to feet, on a plain white background, with the clothing described plainly so it does not fight the face for attention.
Figure 4.2: Anatomy of a character sheet. What it must contain, and why each part matters.
We need two images: the sheet and a close-up portrait made from it. Generate the sheet first, then generate the portrait with the sheet attached as a reference so the face matches, and keep both in a folder named after the character. Those two files are the whole reference bank for this film, and they will be the first two lines of the video prompt.
Reference image prompts
@Image1, the character sheet (16:9, Nano Banana Pro or
equivalent):
Character turnaround reference sheet in the style of a
modern 3D animated feature film, a stylized cartoon
character with softly exaggerated proportions and not a
photograph, 16:9, plain pure white background. The same
eight-year-old girl shown three times side by side at the
same scale, feet on the same baseline, evenly spaced: a
front view on the left, a three-quarter view in the middle,
a right profile on the right. Full body from head to feet in
every view, standing relaxed, arms at her sides except in
the front view where she holds a clipboard against her
chest, and a neutral, serious expression in all three.
Penny: light brown curly hair in a round puff, a round face,
freckles across her nose, big dark eyes. She wears an
oversized white lab coat with the sleeves rolled twice over
a yellow t-shirt, clear safety goggles pushed up into her
hair, denim shorts, one red-and-white striped sock and one
blue-and-white striped sock, and white sneakers. The three
figures are identical in face, hair, clothing and
proportions. Soft, even studio light, no cast shadows on the
background, no text, no labels, no borders, no props other
than the clipboard.
@Image2, the close-up (16:9, generated with @Image1 attached
as a reference):
Using the attached character sheet as the reference for her
face, a close-up portrait in the same stylized 3D animated
style, 16:9, of the same eight-year-old girl, Penny,
front-facing and centered, framed from the collar up: light
brown curly hair in a round puff, freckles across her nose,
big dark eyes, clear safety goggles pushed up into her hair,
the collar of an oversized white lab coat over a yellow
t-shirt. A neutral, deadly serious expression, plain pure
white background, soft even light, sharp focus on the eyes,
no text, no labels.
(A reserve three-quarter portrait for the @Image3 slot, plus
the file names and the judging checklist for sheets, are in
Character_Seed_Prompts_Penny_and_Moe.md.)
@Image1: the character sheet, generated from the prompt above.
@Image2: the close-up, generated from the prompt above.
The identity anchor
Attaching a reference is not the same as using it. Current models accept a stack of images, videos and audio clips per generation, and each one has to be given a job in the prompt, or the model guesses. The job we want is identity, so the first line of the prompt says so in plain words: these two images are Penny.
Then come the constraints, and they matter more here than anywhere else in the series. “Do not alter facial proportions,” “hair, goggles and coat must match the references,” and “no identity drift” are the phrases that stop the model from ageing her, straightening her hair, or quietly losing the goggles halfway through a shot. Figure 4.3 shows what drifts when the anchor is missing and the lines that hold it.
Figure 4.3: The identity anchor. What drifts without it, and the three lines that hold it.
One more choice belongs here. In Article 1 we would have locked a first frame, but for acting a first frame is a trap: it fixes the pose and the expression, and the model then animates cautiously away from it. References give identity without locking the frame, so the model is free to act, which is the whole point of today.
Acting is beats
Directors do not ask actors to “be sad”; they give them beats. A beat is one small unit of intention (listen, decide, deliver, react), and a scene is a chain of beats with a change of mind somewhere in the middle. Actors have worked this way since Stanislavski, and it is exactly the structure a model needs, because “she is serious” is a mood and “she clicks her pen and slides the glass across with two fingers” is a beat.
Our scene has four beats and a turn. Penny presents the sample (beat one), Gus tastes and deliberates (beat two), Gus delivers the verdict (beat three), and both react (beat four), and the turn is that the child is the authority and the adult is the nervous candidate. Figure 4.4 lays the beats on a timeline with the micro-actions that carry them.
Figure 4.4: Anatomy of a timeline prompt. Four beats, each with a face, a hand and a voice.
The listening beat. Half of acting is reacting, and models forget it unless told. When Gus deliberates, Penny holds the pen ready and does not blink; when Gus delivers, Penny writes. Write the listener’s beat in the same window as the speaker’s, or the listener will fidget, smile, or drift out of character.
Dialogue rules
Dialogue is where a generated scene most often falls apart, and it falls apart for boring reasons: the line is too long, nobody knows who is speaking, or the words are not tied to an action. Current models render speech and lip movement natively and do it well when the prompt follows five rules, shown in Figure 4.5.
Figure 4.5: Dialogue rules. Five habits that make generated speech land.
Our film has two lines, six words and two words, and that is not timidity; it is arithmetic. Twenty-five seconds holds about thirty spoken words if nobody does anything else, and our characters have to pour, sniff, sip, deliberate, write and hide a grin. As a general rule, cut dialogue until the beats have room, then cut one more line.
How the model reads it
The references come first, before the subject sentence, because the model needs to know who Penny is before it reads what she does. After that the prompt follows the pattern from the first three articles: subject and action, the look header, time windows, audio, constraints. The new element is the voice, which is described in the audio line the way a face is described in the character header.
Voice as design. “Penny’s voice is small, precise and serious; Gus’s voice is low and warm with a gravelly edge” does two jobs: it tells the model what to synthesize and it stops the two voices sounding alike. Native lip sync follows the quoted words, so keep every quoted word pronounceable and short.
Get Micheal Lanham’s stories in your inbox
Join Medium for free to get updates from this writer.
Text-to-video with references, not image-to-video. We are not locking a first frame today, for the reason above. The references carry identity, the character header carries design, and the frame is free to move with the beats. Figure 4.6 shows the three ways a character can perform and what each one adds, so you know where this article sits.
Figure 4.6: The performance ladder. Three ways to make a character act, and what each one adds.
The workflow
The loop gains two steps at the front, design and references, and the shot spec becomes a beat table.
- Design. Write the five readable lines for each character. This is the character header.
- References. Generate the sheet, then the close-up from the sheet. Two files in a folder named Penny.
- Beats. Four windows, each with a face, a hand and a voice line, and the listener’s beat in the same window.
- Prompt. References first, then subject and action, the look header, the windows, audio with both voices described, constraints with the anchor lines.
- Generate and judge. Twenty-five seconds, 720p, audio on, two reference images.
Window: 0 to 6 s
- Penny (face, hands, voice): Unblinking; pours, slides glass, clicks pen; “Sample seven. Describe the flavor profile.”
- Gus (face, hands, voice): Watches, wary; hands flat on the table; silent
- Camera: Locked two-shot, 35mm
Window: 6 to 14 s
- Penny (face, hands, voice): Pen ready, does not blink; silent
- Gus (face, hands, voice): Eyes to ceiling, one eyebrow; lifts, sniffs, sips; a long silent pause
- Camera: Slow drift to favor Gus
Window: 14 to 20 s
- Penny (face, hands, voice): Waits; pen poised
- Gus (face, hands, voice): Grave; sets glass down, folds hands; “Nobel Prize.”
- Camera: Holds on Gus, Penny in frame
Window: 20 to 25 s
- Penny (face, hands, voice): No smile; writes; tiny fist pump below the table
- Gus (face, hands, voice): Grin hidden behind the mustache
- Camera: Settles back to the two-shot
Here is the prompt.
The generation prompt
Settings
Model: Seedance 2.5, text-to-video with reference images
References: @Image1 the Penny character sheet, @Image2 the
Penny close-up
Duration: 25 seconds
Aspect ratio: 16:9
Resolution: 720p
Audio: on
Seed: any (write it down if you like the result)
Prompt
@Image1 and @Image2 are Penny, an eight-year-old girl with
light brown curly hair in a round puff, freckles, big dark
eyes, an oversized white lab coat over a yellow t-shirt,
clear safety goggles pushed up into her hair, and mismatched
striped socks. In a sunlit kitchen, Penny runs a deadly
serious taste test of her lemonade on her grandfather Gus, a
big man in a brown cardigan with reading glasses on a cord
and a thick grey mustache, who sips it, deliberates like a
judge, and delivers his verdict.
Look: 35mm, shallow depth of field, 24 fps, fine 35mm film
grain, soft morning window light, warm and softly
contrasted, natural skin tones, a real family kitchen with a
wooden table, not a commercial.
[0 to 6 seconds] Locked-off two-shot, eye level, Penny on
the left standing on a kitchen chair, Gus seated on the
right with his hands flat on the table. Penny pours cloudy
lemonade from a glass beaker into a tumbler, slides it
across the table with two fingers, clicks her pen over a
clipboard, and says, clinical and flat: "Sample seven.
Describe the flavor profile." She does not blink. Gus
watches the glass, wary.
[6 to 14 seconds] The camera drifts slowly to favor Gus,
keeping Penny in frame. He lifts the tumbler, sniffs it like
a wine judge, sips, and deliberates: eyes to the ceiling, a
slow chew of nothing, a long pause, one eyebrow rising.
Penny holds the pen ready above the clipboard and does not
move.
[14 to 20 seconds] Gus sets the glass down with ceremony,
folds his hands on the table, looks Penny in the eye, and
says, grave and slow: "Nobel Prize."
[20 to 25 seconds] Penny nods once, writes on the clipboard
without a flicker of a smile, then makes a tiny fist pump
below the edge of the table. Gus hides a grin behind his
mustache. The camera settles back to the two-shot and holds.
Audio: lemonade pouring into glass, the click of a pen, a
wooden chair creaking, birds outside the window. Penny's
voice is small, precise and serious; Gus's voice is low and
warm with a gravelly edge. Only the two quoted lines are
spoken. No music.
Constraints: one continuous shot with no cuts, exactly two
people, Penny's face, hair, goggles, coat and socks must
match @Image1 and @Image2 exactly, do not alter her facial
proportions, no identity drift, Penny never smiles, no
readable text on the clipboard, only the two quoted lines of
dialogue, no extra limbs, no jitter.What we generated this with. Seedance 2.5, because it binds reference images to a named character, renders speech with native lip sync, and keeps a face stable for a full take. The reference syntax (
@Image1) and the time windows are the only vendor-specific parts; other current models take references through their own reference or ingredients features and read the same prompt. One host rule shaped Penny's design: the Seedance endpoint we used refuses reference images that contain photorealistic faces, even generated ones, so her sheet is a stylized 3D character, and on a host that accepts faces a photorealistic sheet binds exactly the same way. Twenty-five seconds at 720p costs roughly five and a half to twelve dollars depending on the host, plus two reference images.
The finished film: 25 seconds from the single generation above, with no edits. This preview is silent. Watch it with sound.
Judging the result
Freeze the first frame, a middle frame and the last frame and put them next to the sheet. Then play it with the sound up and watch the mouths.
- Penny’s face drifts, or she ages, or the goggles vanish. Fix: add a third reference (a three-quarter close-up cropped from the sheet) and repeat the anchor line at the end of the prompt.
- Penny smiles or giggles. Fix: “Penny never smiles” is in the constraints; add “her expression stays flat and serious in every window” to the character sentence too.
- The dialogue is garbled or runs long. Fix: shorten the first line to “Sample seven. Flavor profile?” and keep the tone words; models articulate short lines better.
- A third person appears, or Gus becomes a different man mid-shot. Fix: keep “exactly two people” and add Gus’s five readable lines to the constraints: “Gus keeps his cardigan, glasses on a cord and grey mustache throughout.”
If Penny is Penny in every frame and the verdict lands with a straight face, you have a character who can carry a film, and a reference bank you can reuse in any future prompt.
Where this goes next
You can now design a character, hold her identity and direct her beats. What we have not done is cut. Every film so far has been one shot, and next time we write eight shots into a single generation, make the cuts land on the beat, and use a man who overslept to demonstrate the oldest editing trick in cinema.
This is Day 4 of 7 in the Learn AI Filmmaking series:
This series is built on my book, Learn AI Filmmaking. Each of these lessons, from the logline to the final cut, is covered there in depth.
Check out my best-selling book AI Agents In Action, 2nd Edition by downloading your copy here. Sponsored by Anaconda Inc.
