AI · Audio

Documents in,
audio out.

Turn any PDF or article into a German-language podcast or audiobook — entirely on local hardware, using voice cloning and local LLMs. No external API, no cloud TTS service.


What it does

Two modes, one pipeline.

// PODCAST

Upload a PDF or paste a URL. The system extracts the content, generates a structured script as a dialogue between two named speakers, and synthesises the audio using voice cloning. The result is a conversational MP3 that sounds like a radio discussion of the original document.

// AUDIOBOOK

The original text is cleaned by the LLM — headers, page numbers, and metadata removed — and then read aloud by a single voice. Two synthesis threads run in parallel to keep processing time reasonable for long documents. The output is a complete narration of the source material.


Architecture

Three-stage pipeline
across two machines.

The system is split across two dedicated servers. One handles the web interface and text-to-speech synthesis using a voice cloning model. The other runs the language model pipeline responsible for content extraction and script generation.

For podcast mode the document goes through three stages: one model extracts and chunks the raw content, a second generates a structured script as a dialogue between two speakers, and the TTS engine synthesises each line using the corresponding voice clone. Explicit model unloading between stages keeps GPU memory usage predictable.

TTS hardware
Dedicated NVIDIA GPU server
LLM hardware
Dedicated AMD GPU server
TTS model
Qwen3-TTS (voice cloning)
Script models
Two open-source LLMs, staged
Output format
MP3, streamed or downloaded
Access
HTTPS, login required

Voice Cloning

Three voices,
cloned from audio samples.

Voice cloning uses short WAV reference samples — a few minutes of recorded speech per voice — to reproduce the speaker's characteristics. New voices can be added at any time by extracting clean audio and running an automatic transcription step to generate the reference text.

The TTS model unloads itself from GPU memory after a period of idle time, leaving the card available for other tasks such as image generation.

// MALE

Sven

Primary male voice. Used as default moderator in podcast mode and the default narrator for audiobooks.

Voice cloning German
// FEMALE

Anja

Female voice. Used as the second moderator in podcast mode, providing the conversational counterpart.

Voice cloning German
// MALE

Chris

Secondary male voice. Available as an alternative narrator or moderator when a different voice character is preferred.

Voice cloning German

Challenges

What made this
harder than expected.

New GPU architecture

Unsupported compute target at time of setup

The TTS server uses a very recent NVIDIA GPU. At the time of setup, the standard PyTorch packages did not include support for this GPU's compute target. Getting the CUDA stack working required a specific build of PyTorch that included the new architecture.

Blackwell / sm_120 Custom CUDA build
Thinking models and empty responses

Token budget consumed before visible output

Thinking-mode language models consume tokens internally before producing output. Several parameters had to be tuned to prevent the LLM from returning empty results when the internal reasoning exhausted the budget before any text was written.

Token budget tuning Thinking mode
Long document truncation

Audiobooks cut off at the halfway point

A conservative character limit in the service configuration was silently truncating long PDFs at the midpoint. The limit was increased significantly once the cause was identified. The fix required only a configuration change, not a code change.

Service configuration Character limit

Deployment

Ansible playbook,
served over HTTPS.

The application is deployed through an Ansible playbook from the central management server. The web server runs as a systemd service with all configuration injected via environment variables — model names, paths, secrets, and resource limits included.

A reverse proxy terminates HTTPS using a Let's Encrypt certificate obtained via the Cloudflare DNS challenge. A personal library stores all generated jobs with inline playback, download, and drag-and-drop reordering. On mobile, swipe-to-delete works in the iOS Mail style.

Ansible systemd Nginx + Let's Encrypt Cloudflare DNS