One Script, Five Dialects: How to Turn a Screenplay Into AI Video Prompts for Any Tool

Runway wants physics. Veo wants JSON. Kling wants beats. Here is how to stop rewriting the same scene four times.

Last month I spent an entire afternoon on a single establishing shot. Empty street, one man, a light turning on in a window that should not be lit. Four lines of screenplay. I ran it through four different video models and got back four unrelated pieces of footage: one with a car that did not exist, one where the man teleported halfway down the block, one that ignored the light cue completely, and one that nailed the mood but changed the aspect ratio without asking. Same scene. Same intent. Zero consistency.

The problem was not the models. The problem was that I had written one prompt and pasted it into four boxes, assuming “AI video” was one language. It is not. Runway, Kling, Veo, and Sora each parse a scene through a different mental model, and if you do not translate for that, you get four different guesses at what you meant. That is the one-sentence answer to the question everyone runs into eventually: you cannot reuse a prompt across tools, you have to extract what actually matters in the scene and rewrite it in each tool’s own grammar.

I run ScreenWeaver, a screenwriting and previsualization tool, so this is not an abstract complaint. It is the daily friction of turning a script into pictures.

Why don’t AI video prompts work the same across tools?

Each model was trained to solve a different problem, so each one “listens” for different signals in your text.

Kristopher Dunham laid this out clearly in a prompting guide published in late 2025, and the underlying framework still holds in 2026, even though one of his four examples does not: Runway behaves like a kinetic sculptor obsessed with physics and camera movement, Kling behaves like an audio-visual choreographer that generates sound and picture together, Veo behaves like a rendering engine that wants structured data and reference images, and Sora behaved like a physics simulator that wanted cause and effect spelled out. Behaved, wanted, past tense. OpenAI shut Sora down, more on that below.

That is not marketing language. It shows up directly in what each tool rewards.

Runwayresponds to force and weight. “A car crash” gets you nothing. “A heavy sedan impacts a concrete barrier, the hood crumples with high resistance, glass shatters and carries forward with momentum” gets you a crash that looks like it has mass.
Klingresponds to timeline scripts with beat markers, because it is generating audio and video in the same pass. A prompt structured as- Beat 0-4s: wide shot, market noiseand- Beat 4.5s: SFX: metallic thudwill sync a sound to an action. A plain paragraph will not.
Veoresponds well to structured, almost JSON-like prompts, separating camera, lighting, subject, and audio into distinct fields, plus reference images it calls “ingredients.”
Soraused to respond to causal chains: not “the glass spills,” but “the elbow knocks the glass, it tips on its base, momentum carries the liquid past the rim, it shatters on impact.” That dialect still matters as a way of thinking, even with the tool gone.

Four different scenes, same source material, four completely different vocabularies.

What actually changes between the tools?

Here is the same ten seconds of screenplay action translated into each dialect, condensed from that comparative breakdown:

| Tool | What it wants | Fails on |
|---|---|---|
| Runway | Forces, weight, camera verbs (dolly, truck, boom) | Vague verbs, no physics language |
| Kling | Timed beats, paired audio cues, short dialogue lines | Long monologues, no beat markers |

| Veo | Structured fields (camera, lighting, subject, audio), reference “ingredients” | Prose without explicit sound design |
|---|

| Sora (discontinued) | Cause-and-effect chains, physics notes | Descriptions that skip the “why” |
|---|

None of this is a criticism of any one tool. It is just four different compilers for the same source language, and a screenplay is source code the models were never trained to read directly.

That “discontinued” tag is not filler. OpenAI announced on March 24, 2026 that it was shutting Sora down after operating costs ran near a million dollars a day against a fraction of that in revenue, and a widely reported Disney character-licensing deal fell through. The consumer web and app went dark on April 26, 2026. The API follows on September 24, 2026. Anyone whose only asset was a folder of Sora-specific causal-chain prompts just watched that asset become worthless overnight. Anyone who had kept the scene’s facts written down separately from the prompt lost an afternoon rewriting a template, not a project.

How do you translate one scene without rewriting it from zero every time?

The trick is to stop treating the prompt as the source of truth. The scene is the source of truth. The prompt is just one compiled output of it.

Before touching any tool, pull five things out of the scene, in plain English, and keep them attached to that scene permanently:

Subject and action.Who is doing what, in the fewest words that still carry the intent.
Camera intent.Not “cinematic shot,” but the actual move: does the camera hold still while the subject moves, or does it travel with them?
Lighting and mood.One or two adjectives that would survive translation into any tool’s vocabulary (harsh, warm, flat, high-contrast).
Sound cues.Even a placeholder like “distant traffic, one sharp sound at the turn” matters, because Kling and Veo both expect audio direction and Runway and Sora will ignore it, which is fine, you just drop that field for them.
Physical rules that must hold.Anything that would look wrong if it drifted: does the door stay solid, does the character keep the same jacket, is gravity normal or not.

Once you have those five things written down once, per scene, you are not writing four prompts from scratch. You are running one small set of facts through four templates. What changes the outcome is not a magic phrase, it is discipline about separating the scene’s truth from its four possible renderings.

Go back to that street scene from the start of this piece. Written down once, the five facts look like this: subject and action, a man walks, then freezes; camera intent, static wide shot, no movement until the freeze; lighting and mood, cold, high contrast, one practical light source; sound cues, quiet street ambience, then nothing; physical rules, the light in the window switches on abruptly and stays on. For Runway, that becomes a paragraph about weight and stillness, the man’s shoes on wet asphalt, his body decelerating to a stop. For Kling, it becomes a two-beat timeline, Beat 0-6s: static wide, man walks then Beat 6s: freeze, light turns on inside window, with an audio line for the moment the light clicks. Same five facts, two completely different documents, and neither one had to be invented twice.

This is the exact seam I kept hitting while building ScreenWeaver’s workflow engine. The screenplay stays the source of truth, and a scene’s frame direction and video direction get generated from that scene’s own context, its characters, its location notes, its established look, rather than from a prompt typed fresh each time. Swap the video model behind a workflow and the underlying scene facts do not change, only the template that renders them does. It is the same idea Dunham describes for solo creators juggling four tools by hand, just wired into the pipeline instead of kept in your head.

The prompt is not the scene. The prompt is a translation of the scene into whatever dialect the model in front of you happens to speak.

We used a version of this discipline on Lost Garden, an animated series pilot that ended up a festival finalist at the AI London Festival. Different shots in that pilot needed different strengths: some needed tight lip-sync on dialogue, others needed a camera move that held weight through a fast pan. Writing “the man walks into the room” once and hoping it worked everywhere was never going to hold up across that range. Translating the same shot description into each tool’s own terms, on purpose, every time, was the only thing that did.

Is it worth building this yourself, or should you just trust a tool?

Learn the manual translation first, even if you plan to automate it later. If you do not understand why Kling wants beat markers and Veo wants structured fields, you cannot tell when an automated pipeline is quietly producing bad output. You will just see “wrong result” and not know which layer failed: the scene facts, the template, or the model.

Once you can do the translation by hand and recognize a good result from a mediocre one, automating the repetitive part is just saving yourself typing. Skipping the manual stage and trusting a black box from day one is how people end up with four inconsistent clips and no idea which knob to turn.

The cheapest way to start is not a new tool at all. It is a spreadsheet or even a plain text file, one row per scene, five columns for the five facts. Fill it in while you are still writing, before you open a single video model, and treat it as part of the screenplay rather than a separate production document. When a shot needs a second tool, or the first tool you liked gets discontinued, you are translating from that row, not staring at a blank prompt box trying to remember what the scene was supposed to feel like.

A short FAQ

Do I need to learn every tool to make a film?

No. Pick the tool whose strength matches the shot. Runway for anything where weight and camera movement carry the shot, Kling for dialogue and music-driven scenes, Veo when you need tight reference-image consistency across a series. Most independent productions end up using two of the tools still standing, not all of them.

Can I just use one tool for everything?

You can, and plenty of people do, but you will hit a shot that tool cannot do well. Knowing the second dialect before you need it is cheaper than learning it under deadline.

What happens to my prompts when a tool shuts down?

Ask anyone who built their workflow around Sora. OpenAI discontinued the app and web experience in April 2026, with the API following in September. If your only asset was a stack of Sora-specific prompts, that asset is gone. If your asset was the scene’s facts, subject and action, camera intent, lighting, sound, physical rules, you spend an afternoon writing a new template for whatever tool replaces it.

Does this get easier as the models improve?

The syntax will keep shifting, version to version, and tools will keep disappearing entirely. What will not change is the underlying habit: separate the scene’s facts from the prompt that renders them, and you can adapt to whatever the next model wants, or survive whatever the last one did, without starting over.