Remake: How Spoken Lines and Music Are Read From the Clip, and What Ends Up in Your Episode

Quick answer: Every shot of the clip is listened to twice: first by a speech recogniser that answers ‘is anyone speaking, and what did they say’, then by the model that describes the shot, which is given the recognised words to work from. Lines are therefore quoted, not guessed from lip movement, and a shot where nobody speaks is written as silent. If the clip has music, the read notes its character, and that becomes the tone the episode’s own score is written in: it is planned from the episode’s storyboard — usually one to three pieces — and mixed in under the voices, as in any drama. With Remake the story, the dialogue language is chosen on the upload screen and fixed once the read starts: lines are written in it, translated if the source speaks another. A line the recogniser could not make out is simply missing; compare with your clip and add it to the shot in the editor once the storyboard is generated. In SceneMixer, a line the recogniser could not make out is simply missing from the read.
Still from “Sky Rider's War”, an AI short drama generated end-to-end by SceneMixer (10 characters, 1 episode)
SampleSky Rider's WarShot on Film1 episodeProduced end-to-end by SceneMixer from the scriptWatch the episodes

After a Remake the story read, open the episode in Step 3 and read its outline: each scene block lists its lines in speaker-colon form. Swap keeps the clip’s own sound and lines, so this page does not apply to it.

How lines are heard

Speech recognition runs on each shot’s own audio. What it hears is handed to the shot description as ‘these words were spoken here’; the describer then names the speaker from the faces and writes the line into the shot. Music under speech is handled; shouting over a crowd, heavy accents and very quiet lines are where it fails, and those are left out rather than invented. Lines then appear in the outline like a script’s: Write Dialogue So It Is Picked Up.

A storyboard segment: location, time and weather fields, then numbered shots with durations, asset chips and spoken lines
Each shot is listened to; what was heard is quoted into the read and ends up as lines in the outline.

Why quote, not guess

Left to itself, a vision model asked ‘what is said here’ will sometimes hear and sometimes not, on the same shot, and will invent plausible lines when unsure. Recognised words pinned into the description stop both; the same words are used every time the shot is described.

Music

The read asks once whether the clip has music and what kind. If it does, that description becomes the tone the episode’s score is written in, so your remake sounds like the same kind of film; the source track itself is never used. The clips are then generated without music of their own and the score is mixed in under the voices, exactly as in a drama. When an episode is not being scored — an older episode, or one whose plan could not be made — the video model gives those clips their own music instead, as before. Ambient sound from the clip is noted but not carried. Background music in an AI short drama.

Dialogue language

Chosen on the upload screen when you pick Remake the story, defaulting to your interface language (Chinese on the Chinese site), and fixed once the read starts. Lines heard in another language are translated into it. It cannot be changed afterwards for a remake, which is why Step 1 does not offer the dropdown. Swap has no such setting because it keeps the clip’s own voices; if you switch a swap project to the story later, its lines are in your interface language. After the Read.

When a line was missed or wrong

The outline itself cannot be edited on SceneMixer, and the storyboard copies its lines word for word. Generate the storyboard, then open the segment where the line belongs and add or fix it there, with its speaker; a line given to the wrong person is fixed the same way. Editing is free. Generate the Storyboard.

In the clipIn the readIn your episode
A clear spoken lineQuoted, speaker namedA line in the outline
Nobody speaksShot marked silentNo line
Unclear speechLeft outAdd it in the segment by hand
MusicCharacter notedTone for the episode’s own score, mixed under the voices
Ambient soundNotedNot carried; the storyboard writes its own

FAQ

Why is a line missing?

The recogniser could not make it out. Compare with your clip and add it in the segment once the storyboard exists.

Are the source voices used?

Not when you remake the story: voices come from the cast's voice settings, as in any project. Swap keeps the clip's original sound.

Can the remake speak a different language than the source?

Yes, with Remake the story: pick the dialogue language on the upload screen and the lines are translated at the read.

Is the source music reused?

Not when you remake the story. Its character becomes the tone of the episode’s own score, which SceneMixer plans from the episode’s storyboard — usually one to three pieces — and mixed under the voices. Swap keeps the clip’s original sound, music included.

Why does a crowd shout get no line?

Group noise is treated as background sound, not dialogue.

Heard, quoted, written

What was said in the clip is what your episode says.

Start a remake

Updated November 15, 2026