Overview

Audio8-TTS-Preview-0.6b is a 0.6B-parameter multilingual text-to-speech model built by Audio8 that generates speech from text and performs zero-shot voice cloning. The model uses a DualAR architecture with a 24-layer slow AR transformer (896 width, 14 attention heads, 2 KV heads) and a 4-layer fast AR transformer that work together to predict semantic tokens and then acoustic codebooks. The bundled codec operates at 44.1 kHz with 2,048 samples per model frame, producing approximately 21.5 frames per second, and supports up to 2,048 packed text/audio context positions. The model runs on transformers>=4.57.0 and requires Python 3.10 or newer with a CUDA-capable GPU. Despite being the smallest model in its comparison class, it achieves state-of-the-art-class performance on multilingual benchmarks with only 601,159,424 parameters (excluding the codec), making the single most important thing to know that this model delivers competitive quality at a fraction of typical TTS model size.

Best use cases

Zero-shot voice cloning for customer service automation. This model excels at generating natural speech in a customer's own voice without requiring training data or fine-tuning. Provide a reference audio clip with matching transcript, then generate responses in that voice. The DualAR architecture's ability to encode speaker characteristics from minimal examples makes this practical for creating personalized chatbots or IVR systems where maintaining voice consistency matters more than absolute quality.

Multilingual content localization with voice consistency. The model supports 11 languages—Cantonese, Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Polish, and Spanish—and maintains voice characteristics across languages. Use this for localizing marketing videos, audiobooks, or educational content when you need consistent narration across multiple languages without hiring different voice actors.

Low-latency speech synthesis on edge devices. At just 0.6B parameters, this model fits on consumer GPUs and mobile devices where larger 4-8B parameter TTS systems cannot run. The compact size combined with the 44.1 kHz codec and ~21.5 frames-per-second generation rate makes this suitable for real-time conversational AI, accessibility tools, or embedded applications with strict latency budgets.

Hard phonetic and tonal language synthesis. The model achieves best-in-class English word error rate (1.506%) on Seed-TTS and handles challenging Chinese tones and Hard Cantonese variants (11.510% CER on hard Mandarin). Use this when synthesizing languages with complex prosody requirements where competing models struggle with naturalness or phonetic accuracy.

Development and prototyping before model scaling. The compact size and Apache 2.0 license make this suitable for rapid prototyping of TTS features with lower compute costs. Validate voice cloning, multilingual generation, or integration patterns before committing to larger, more expensive models in production pipelines.

Limitations

This is a Preview checkpoint with intentionally limited language coverage. The documentation explicitly states that 11 languages are recommended and broader multilingual coverage plus Chinese dialect support are planned for future releases. Do not expect performance on languages outside the supported list, and avoid depending on this model for languages with currently weak evaluation coverage.

Voice cloning stability depends heavily on reference audio quality. Long, noisy, or incorrectly transcribed reference clips can degrade speaker similarity and generate unstable output. The reference transcript must match the spoken content exactly—mismatches cause failure.

The model generates speech that sounds natural enough for impersonation and misinformation. The documentation warns that generated speech can be misused to impersonate individuals or spread false information. You must obtain explicit consent before cloning a voice and clearly disclose synthetic audio to listeners. Legal liability for misuse remains yours.

GPU acceleration is required. The model recommendation for Python 3.10+ and CUDA-capable GPU means CPU-only inference is slow or impractical. The exact VRAM requirement is not specified, though inference with bfloat16 precision is standard; estimate based on the 0.6B parameter count plus codec overhead.

Evaluation methodology differs across benchmarks. The documentation notes that different normalizers and evaluators make cross-project comparisons "reference comparisons rather than a strictly matched ranking." Claims about outperforming competitors should account for methodological differences rather than assuming absolute superiority.

This is a preview release, not production-ready. The checkpoint comes with all the typical early-stage risks: stability issues, missing features, limited community support, and potential breaking changes in updates. Evaluate thoroughly for your specific use case before deployment.

How it compares

Fun-Audio-Chat-8B is an 8B large audio language model designed for natural, low-latency voice interactions with multimodal input capabilities. Audio8-TTS-Preview should be picked when you need compact size and multilingual support; pick Fun-Audio-Chat when you require a full dialogue system with audio understanding and real-time speech-to-speech capabilities where parameter count and latency matter less than conversational ability.

Higgs Audio v3 TTS is a 4B model supporting 100+ languages with inline control over emotion, style, and prosody. Choose Audio8-TTS-Preview for compact deployment, zero-shot cloning, and strong performance on the 11 supported languages; choose Higgs Audio v3 when you need broad language support, expressive speech control, or don't have tight size constraints.

MisoTTS is an 8B multilingual model emphasizing natural speech quality and speaker control. Audio8-TTS-Preview wins on parameter efficiency and benchmark performance on its supported languages; MisoTTS is better when you need broader language coverage and can allocate more compute.

Spark-TTS-0.5B is a 0.5B LLM-based TTS built on Qwen2.5 for efficiency and simplicity. Both models target compact size, but Audio8-TTS-Preview demonstrates stronger multilingual and zero-shot voice cloning capabilities; pick Spark-TTS if you prefer LLM-based architecture or simpler fine-tuning workflows.

MioTTS-2.6B is a lightweight LLM-based TTS supporting English and Japanese with zero-shot voice cloning. Audio8-TTS-Preview is superior for multilingual use (11 vs 2 languages) and achieves better inference efficiency; choose MioTTS-2.6B only if you specifically target English and Japanese and prefer LLM architecture.

Technical specifications

Architecture: DualAR (Dual Autoregressive) transformer design with separate slow and fast transformer branches. The slow AR transformer (24 layers, 896 width, 14 attention heads, 2 KV heads) predicts semantic tokens at one token per audio frame. The fast AR transformer (4 layers, same width and attention configuration) predicts the frame's 10 acoustic codebooks (4,096 entries each), conditioned on the slow transformer's hidden state and preceding codebook predictions.

Model size: 601,159,424 parameters (excluding the bundled codec). Classified as a 0.6B-parameter model.

Codec: 44.1 kHz neural audio codec with 2,048 samples per model frame, producing approximately 21.5 frames per second. Bundled with the checkpoint; no external codec required.

Context window: Up to 2,048 packed text/audio positions for handling longer sequences.

Training and evaluation:

  • Achieves 1.506% English WER on Seed-TTS (best among compared models)

  • 0.950% Chinese CER on Seed-TTS

  • 11.510% CER on hard Mandarin (Seed-TTS)

  • 3.128% error rate on English CV3 multilingual evaluation

  • 3.205% error rate on Chinese CV3 multilingual evaluation

Framework and libraries: Transformers library (>=4.57.0, <5), PyTorch (>=2.5.0), torchaudio (>=2.5.0), soundfile (>=0.12), safetensors (>=0.4). Loads with trust_remote_code=True due to custom Transformers code.

Supported languages: Cantonese, Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Polish, Spanish (11 total).

Dtype: bfloat16 on CUDA devices, float32 on CPU. bfloat16 is standard for inference.

License: Apache License 2.0.

Model inputs and outputs

Inputs

  • Text:List of strings containing the text to synthesize. Unlimited length but practical limits depend on context window (2,048 tokens).

  • Reference audio (optional):WAV file path or audio array for zero-shot voice cloning. 44.1 kHz mono or stereo recommended.

  • Reference text (optional):Exact transcript matching the spoken content in reference audio. Required only if reference_audio is provided.

  • Generation parameters:temperature (default 0.8), top_p (default 0.95), top_k (default 50), max_new_tokens (default 1024).

Outputs

  • Audio codes:Semantic and acoustic tokens in compressed form (10 codebooks × 4,096 entries).

  • Waveform:Floating-point audio array at 44.1 kHz sample rate decoded from codes.

  • Waveform lengths:Integer tensor specifying the valid length of each output waveform in a batch.

Getting started

```
import soundfile as sf
import torch
from transformers import AutoModel, AutoProcessor
model_id = "AutoArk-AI/Audio8-TTS-Preview-0.6b"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32

Load processor and model

processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(
model_id,
trust_remote_code=True,
dtype=dtype,
).eval().to(device)

Zero-shot voice cloning

inputs = processor(
text=["Welcome to Audio8 TTS."],
reference_audio=["reference.wav"],
reference_text=["The exact transcript of the reference recording."],
return_tensors="pt",
)
inputs = {name: value.to(device) for name, value in inputs.items()}
with torch.inference_mode():
output = model.generate(
**inputs,
max_new_tokens=1024,
temperature=0.8,
top_p=0.95,
top_k=50,
do_sample=True,
return_dict_in_generate=True,
)
waveforms, waveform_lengths = model.decode_audio(output.codes)

Save output

audio = waveforms[0, : int(waveform_lengths[0])].float().cpu().numpy()
sf.write("output.wav", audio, model.config.codec_sample_rate)

Without voice cloning

inputs_no_clone = processor(
text=["This utterance does not use a reference voice."],
return_tensors="pt",
)
```

Frequently asked questions

Q: Can I use this model commercially?

A: The model is released under Apache License 2.0, which permits commercial use. However, you must clearly disclose when speech is synthetically generated and obtain explicit consent before cloning anyone's voice, or you accept legal liability for misuse including impersonation or fraud.

Q: What GPU VRAM do I need to run inference?

A: The documentation does not specify VRAM requirements. With 0.6B parameters and bfloat16 precision, estimate approximately 1.2 GB for the model weights plus overhead for activations and codec operations. Most modern consumer GPUs (RTX 3060 12GB and higher) should run it comfortably, but exact requirements depend on batch size and context length.

Q: How does this model compare to Higgs Audio v3 TTS for multilingual synthesis?

A: Audio8-TTS-Preview supports 11 languages with best-in-class English performance (1.506% WER); Higgs Audio v3 supports 100+ languages with more expressive prosody control. Pick Audio8 for compact size and strong performance on its 11 languages; pick Higgs Audio v3 if you need broader language coverage or expressive speech control.

Q: What happens if my reference audio doesn't match the reference text?

A: The model expects exact transcript matching. Mismatches cause poor quality output or failure. Ensure the reference text captures everything spoken in the reference audio, including filler words, pauses, and speech patterns.

Q: Can I fine-tune this model?

A: The documentation mentions supervised fine-tuning is supported. See the Audio8 TTS repository on GitHub for implementation details. Fine-tuning code is available in the main repository, but specific framework and dataset requirements are not detailed in this README.

Q: How fast is inference? What batch sizes work?

A: The codec generates ~21.5 frames per second, but the documentation does not report end-to-end latency or practical batch sizes. Real-time or near-real-time performance is likely for short sequences on modern GPUs, but exact timings depend on hardware and sequence length.

Q: Does this model support languages other than the 11 listed?

A: Not reliably. The documentation explicitly states that the 11 recommended languages are the supported set for this preview release. Broader multilingual coverage is planned for future releases. Attempting synthesis in other languages will produce poor-quality output or fail.

Q: Is this model actively maintained?

A: This is a preview release, suggesting active development. Check the Audio8 TTS GitHub repository for the latest updates, bug fixes, and community discussions. Preview status means breaking changes or significant improvements are possible in future versions.