Run Fish Audio Drama 3 Locally: The 48 GB GPU Rule, the 22 GB int4 Build, SGLang-Omni and Docker Setup, Voice Cloning, Director Tags, the License and AWS GPUs

Fish Audio Drama 3 is a new open-weight text-to-speech model, released on October 13, 2026, that you direct like an actor: write [sighs] or <whisper>...</whisper> into the script, give it a note like "gentler, and a bit sadder" for a second take, clone a voice from a few seconds of audio, and have it play up to 16 characters in one scene, in 83 languages. The weights are free to download and free to use commercially under $500,000 of annual revenue. Here is the fact to read before anything else: Drama 3 needs an NVIDIA GPU with at least 48 GB of memory, and nothing else runs it: no CPU, no Mac, no AMD. A 35-billion-parameter model with only 3 billion active per word still has to fit all 35 billion in VRAM. An unofficial int4 build brings the weights down to about 21 GB, which puts 24 GB cards in play, and a single AWS L40S instance happens to have exactly 48 GB. This page covers the real file sizes, the three ways to run it (Fish Speech, Docker, SGLang-Omni), voice cloning, director tags and revisions, the license, and the AWS route.

Ethan, a developer, has been asked by his friend Jake to make the phone repair shop's hold message and a short explainer video voice-over without paying for a studio. Jake wants it warm, with a laugh at the right moment, and he wants to change one word without re-recording the whole thing. Ethan's own PC has a 12 GB graphics card, which is where this story's first problem lives.

New to running AI on your own machine? Our plain-English series on running AI locally explains parameters, quantization and GPU memory in about ten minutes; this page assumes none of it.

What Drama 3 is, in plain English

Fish Audio makes text-to-speech models and runs a paid voice service on top of them. Drama 3 is their new flagship, and unlike most commercial voice models it comes with its weights and a technical report. The "drama" in the name is literal: the model is built for performances rather than announcements. Its shape, in the terms you'll meet in the download:

• 35B-A3B: a 35-billion-parameter mixture-of-experts text backbone (256 experts, 8 active plus one shared, started from Qwen3.5) with about 3 billion active per token. Fast to run, big to hold.
• A codebook decoder turns the backbone's output into sound: 13 layers, 10 codebooks at about 21.5 frames a second, 44.1 kHz audio out.
• FP8 experts, with the rest of the weights in bf16. That is why the official checkpoint is 41.5 GB rather than 70, and why the GPU has to support FP8.
• Trained with reinforcement learning from a single human-preference reward model, which is the part that makes "a bit sadder" work as an instruction.

What it does that the previous model, S2-Pro, did not: edit a finished take by replacing a word while the rest stays as recorded (infilling), redo a take from a note while remembering the previous one, voice a whole scene of up to 16 characters from a script, stream long text as it generates, and carry laughter, crying and whispers through a cloned voice instead of flattening them. Multilingual cloning error rates also dropped. S2-Pro, at 4B parameters, stays supported with the same commands and runs on a 24 GB GPU or even a CPU, which matters for the next section.

Jake: "Thirty-five billion, three billion active. So it's quick?"

Ethan: "Quick on the right card. On one H200 it makes audio five times faster than it plays. The catch is the word 'hold': every one of those 35 billion sits in VRAM whether it's working or not."

Will it run on my machine? The size table

Here is the catch hiding in "3B active". Speed comes from the active parameters; memory comes from all of them. The runtime loads the entire backbone plus the decoder and codec into GPU memory, so the file size, not the active size, decides who can run it:

Three things are ruled out by the maintainers, in writing: CPU inference, Apple silicon through MPS, and AMD through ROCm. Also ruled out is --compile, the torch.compile speed-up that S2-Pro uses. The official 48 GB figure includes room for the activations and the audio decoder; the int4 build's own card says its weights take 20.8 GB against 37.0 GB for FP8, which is how it fits a 24 GB card with a few gigabytes to spare for generation.

Jake: "So my 12 GB card is out, your 12 GB card is out, and the shop's whole budget is a used 3090?"

Ethan: "Or an afternoon on a rented L40S, which costs less than the used card's shipping. We render the hold message once; we don't need the GPU on Tuesday."

Download the weights

The weights are not gated: no license click on Hugging Face, no token for a public download. You still need the Hugging Face CLI and about 45 GB of disk for the official checkpoint, or 24 GB for the int4 one. Fish Speech expects the checkpoint in a checkpoints folder next to the code; SGLang-Omni takes the Hugging Face name and fetches into its cache:

pip install -U "huggingface_hub[cli]"
hf download fishaudio/drama-3 --local-dir checkpoints/drama-3          # 41.5 GB, official
hf download fishaudio/drama-3-int4 --local-dir checkpoints/drama-3-int4  # 24 GB, unofficial int4

One detail from the install guide that saves a confusing error: if a checkpoint directory has no codec.pth, Fish Speech falls back to the S2-Pro codec at checkpoints/s2-pro/codec.pth, so a trimmed download needs S2-Pro downloaded as well. The official Drama 3 repository includes its own codec.pth, so a full download has no such dependency.

Run it with SGLang-Omni: the launch-day server

Fish Audio's own recommendation for anyone who cares about speed is SGLang-Omni, and it's also the only documented path for the int4 build. It installs from the repository and serves an OpenAI-style speech endpoint:

git clone https://github.com/sgl-project/sglang-omni.git && cd sglang-omni
uv pip install --prerelease=allow -v -e ".[fishaudio]"
hf download fishaudio/drama-3
sgl-omni serve --model-path fishaudio/drama-3 --port 8000
# int4 on a 24 GB card:
# sgl-omni serve --model-path fishaudio/drama-3-int4 --port 8000

That command puts preprocessing, the engine and the vocoder on GPU 0. Then, from any terminal:

# plain text to speech
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input": "Thanks for calling. We will be with you in a moment."}' \
  --output hold.wav

# streaming, raw PCM as it generates
curl -N -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input": "Thanks for calling.", "stream": true, "response_format": "pcm"}' \
  --output hold.pcm

The serving numbers Fish Audio publishes for this setup, measured on one H200 with voice cloning on every request: a single stream gets its first audio in about 89 milliseconds and generates at about 7.6 times real time; at 16 concurrent streams it still generates 2.7 times faster than playback; at 64 streams the mean comes down to 1.25 times real time. If you need many streams and a short time to first audio together, the cookbook's answer is to move the vocoder onto a second GPU, which it reports as about 72% more throughput at 256 streams.

Run it with Fish Speech: the command line and the API server

The Fish Speech repository is the reference implementation, with a command line, an API server and a web UI, and it's the right choice when you want every editing and multi-turn feature through its own API. Linux or WSL only; Windows and macOS are off the table for Drama 3 because of the GPU rule.

sudo apt install -y portaudio19-dev libsox-dev ffmpeg
git clone https://github.com/fishaudio/fish-speech.git && cd fish-speech
# with uv (pick cu126, cu128 or cu129 to match your driver)
uv sync --python 3.12 --extra cu129
# or with conda
# conda create -n fish-speech python=3.12 && conda activate fish-speech
# pip install --extra-index-url https://download.pytorch.org/whl/cu129 -e ".[cu129]"
hf download fishaudio/drama-3 --local-dir checkpoints/drama-3

One file in one command, cloning a voice from a reference clip and its transcript:

python fish_speech/models/text2semantic/inference.py \
    --checkpoint-path checkpoints/drama-3 \
    --text "[warm] Thanks for calling Jake's. [laughs] Yes, we fix that one too." \
    --prompt-audio "reference.wav" \
    --prompt-text "The exact words spoken in reference.wav" \
    --output "hold.wav" \
    --device cuda:0

On older GPUs that can't hold it on one card, --device cuda:0,cuda:1 spreads it, and the last device holds the audio decoder and the codec. The API server is the same idea with a port:

python tools/api_server.py --listen 0.0.0.0:8080 --llama-checkpoint-path checkpoints/drama-3 --device cuda:0
curl http://127.0.0.1:8080/v1/health          # {"status":"ok"}

The server's endpoints: POST /v1/tts (the body is msgpack, which the Fish Audio SDK speaks for you), GET /v1/tts/defaults for the model's sampling defaults, /v1/references/add and /v1/references/list to save a cloned voice under an ID and reuse it, and /v1/vqgan/encode and /v1/vqgan/decode for the audio tokens. --api-key adds bearer-token authentication, --max-text-length caps a request, and --half switches to fp16 on cards without bf16. The web UI ships at /ui.

Run it in Docker

The prebuilt image runs both models and includes the web UI. Mount the checkpoints folder, name the checkpoint, publish the port. On an Ada or newer card expose one GPU; on older cards use --gpus all so the decoder can land on a second one:

docker run --gpus device=0 \
    -v ./checkpoints:/app/checkpoints \
    -e LLAMA_CHECKPOINT_PATH=checkpoints/drama-3 \
    -p 8080:8080 \
    fishaudio/fish-speech:server-cuda

Then http://localhost:8080/ui for the interface and http://localhost:8080/v1/health for the check. The image needs the NVIDIA container runtime installed on the host, which on Kali or Ubuntu is the nvidia-container-toolkit package plus a Docker restart. The CPU image exists but only serves S2-Pro; a Drama 3 request on it fails for the reason this whole page keeps repeating.

Directing it: tags, scenes, revisions and edits

This is the part that separates Drama 3 from a reading machine, and it all lives in the text you send. Four kinds of control, from the model card and the release notes:

Voice cloning is a reference clip plus its transcript, passed with the request. A few seconds is enough; the published cloning example uses a 3.9-second clip. A second request with the same reference and the edited text is how editing works through the API: the server treats the earlier audio as context, the way a language model treats earlier turns.

{
  "text": "The quarterly report is due on Thursday, so please finish it by noon.",
  "references": [{"audio_path": "reference.wav",
                  "text": "We asked over twenty different people, and they all said it was his."}],
  "previous_takes": ["The quarterly report is due on Wednesday, so please finish it by noon."]
}

That request changes "Wednesday" to "Thursday" and leaves everything else as it was. A third request with both earlier takes in previous_takes chains a second edit. Fish Audio's numbers for this on a public editing benchmark put its word-error rate below a well-known commercial infill API in both English and Chinese, and three rounds of revision raised emotion accuracy from 55.5% to 58.2% on their multi-turn test; modest numbers, honestly reported, which is more than most voice vendors publish.

Sampling: the Fish Audio SDK defaults are temperature 0.7, top-p 0.7, top-k 30, and the server's /v1/tts/defaults returns them; the throughput tables were measured at temperature 1.0 and top-p 0.9. Long text is chunked automatically at about 300 bytes with at most three speakers per chunk, and every chunk is validated before generation starts, so a malformed speaker tag fails early rather than after a minute of audio.

Jake: "So I can change 'Thursday' to 'Friday' in the hold message without you re-recording it?"

Ethan: "Without me re-rendering it. I send the old take and the new sentence, and it only redraws that word. Your laugh in the middle stays exactly where it was."

The API and the app, if you'd rather not run it

Drama 3 is live in Fish Audio's Text to Speech web tool, the iOS and Android app, and the API, under the model name drama-3. Every tag above works in the API as typed, and it's the same endpoint the previous model used:

curl https://api.fish.audio/v1/tts \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "model: drama-3" \
  -d '{"text": "[warm] Thanks for calling. [laughs] Yes, we fix that one too."}' \
  --output hold.mp3

List price is $50 per million characters, with an 80% launch discount to $10 per million from October 13 to November 12, 2026. A million characters is roughly 150,000 words of script, so a repair shop's hold message, a product video and a month of social clips sit well under a dollar at the launch rate. The editing tools in the web app were invite-only at launch, with an editing API announced as coming; through the open weights, the previous_takes mechanism above is available today.

The license, in plain English

Drama 3 ships under the Fish Audio Community License v1.0, a new license; S2-Pro stays under the older Research License. What it means for you:

• Free for research and non-commercial use, and free for commercial use if you and your affiliates have under US$500,000 in annual revenue. Cross the line and you must notify Fish Audio and move to a written agreement after a transition period; crossing it is not a breach.
• No hosted service at any revenue level: you cannot offer the model to third parties so they can generate speech from their own text or voices, through an API, a web app or software you operate, without a separate license. Generating speech for your own content, your own app's characters or an assistant's replies is fine, and so is a studio or freelancer delivering finished audio to clients.
• No cloning a real person without consent, no removing watermarks or provenance signals, and no using it to train another speech or foundation model.
• Attribution: "Built with Fish Audio" on a related website, UI or about page, and the notice file kept with any copy you distribute.
• You own the outputs, and you own the derivative models you make, including fine-tunes and quantizations like the int4 build, under the same terms.

For Jake's shop, a solo developer, or a startup under the line, that is a yes for anything that isn't a voice-generation service for other people. Read the license file in the repository before you ship a product on it; the hosted-service clause is the one that changes answers.

Renting the right GPU on AWS

The 48 GB rule maps onto AWS unusually cleanly. The g6e family carries one NVIDIA L40S per GPU with 48 GB, so a g6e.xlarge is the smallest instance that runs the official checkpoint, and the L40S is an Ada card, so the FP8 experts run natively. The g5 (A10G) and g6 (L4) families have 24 GB per GPU and fit the int4 build, with the same tightness a desktop 24 GB card has. The p4d A100 40 GB is below the floor; the 80 GB A100 and H100 instances are comfortable and expensive.

The setup is the Linux one above on an Ubuntu or Amazon Linux Deep Learning AMI, which already has the driver and the container toolkit. Two habits save money: stop the instance the moment the render is done, because the meter runs while the GPU idles, and keep the 45 GB of checkpoints on a volume you don't delete, so the next session doesn't start with the download. Our Hugging Face on AWS guide has the instance sizes and per-hour prices, so this page won't repeat them. Drama 3 isn't on Amazon Bedrock; this is a run-it-yourself model on AWS, or a call to Fish Audio's API from Lambda if you'd rather not own the GPU hour.

Drama 3 or S2-Pro, and the open voice field

If your card is under 48 GB and you want to stay inside Fish Audio's tooling, S2-Pro is the answer: same install, same server, same tags for emotion and prosody, 4B parameters, a 24 GB card or a CPU, and a ROCm image for AMD. What you give up is editing, multi-turn revision, the strongest laughter-and-whisper delivery, and the lowest cross-lingual cloning error rates. What you gain is a model you can actually run tonight.

Against the closed voice APIs, the honest comparison is the one Fish Audio makes itself: lower infill error than a leading commercial editing API on a public test, and a serving stack you can inspect. Against other open voice models, the 48 GB floor is the real difference; most open text-to-speech models fit a 12 GB card, and Drama 3 is a different class of machine, closer to running an open LLM than a classic TTS model.

When it doesn't work

Ethan rented a g6e.xlarge for two hours, pulled the official checkpoint onto a volume he kept, and rendered the hold message with a laugh after "yes, we fix that one too." Jake asked for "Thursday" to become "Friday" the next morning; one request with the old take attached, and the laugh didn't move. Jake's verdict: "So the actor lives in the cloud and we pay by the hour." Ethan said that was the cheapest voice talent he'd ever hired.

Revision note. Written October 15, 2026, two days after the Drama 3 weights and the v3.0.0-beta tooling appeared, with the int4 build and the SGLang-Omni cookbook as they stood. "Cut your coat according to your cloth," says the proverb; here the cloth is 48 gigabytes of VRAM.