trawl
Back to Playbooks
Academiceasy
podcaststranscriptionapitutorial

How I Transcribed 50 Podcast Episodes for a RAG Pipeline

Trawl Team·

I needed transcripts from 50 podcast episodes for a RAG pipeline. The project was a research assistant that could answer questions about everything a specific podcast host had said over the past year. The problem: podcasts live behind RSS feeds, Apple links, and Spotify URLs — and fewer than 1% include transcript files.

I spent a weekend wiring together PodcastIndex for search, feedparser for RSS, yt-dlp for audio download, and Whisper for transcription. It worked, but it was brittle and slow. Then I collapsed the whole thing into a few Trawl API calls.

How the pipeline works

Trawl's podcast engine has three layers:

RSS transcript detection (free, instant) — About 1% of episodes include transcripts via the Podcasting 2.0 <podcast:transcript> tag. When Trawl finds one, it downloads and parses it immediately. No compute cost, no delay.

AI transcription (accurate, async) — For the other 99%, Trawl downloads the audio and runs speech-to-text at 3-4% word error rate across 99+ languages. A typical 1-hour episode completes in 1-3 minutes.

Speaker diarization (Pro+ tier) — For interview podcasts, the Pro tier adds speaker identification — labeling who said what throughout the transcript.

Try it

The full pipeline

This is the script I use. It searches for a podcast, browses episodes, resolves the audio URL, and submits transcription jobs.

from trawl import Trawl
import time

client = Trawl(api_key="trawl_your_key")

# Step 1: Search for the podcast
results = client.podcasts.search(q="All-In Podcast")
podcast = results.feeds[0]
print(f"Found: {podcast['title']} (ID: {podcast['id']})")

# Step 2: Get recent episodes
episodes = client.podcasts.episodes(podcast_id=podcast["id"], max_results=10)
print(f"Found {len(episodes.items)} episodes")

# Step 3: Transcribe each episode
for ep in episodes.items:
    print(f"\nTranscribing: {ep['title']}")

    # Submit the transcription job
    job = client.podcasts.transcribe(
        audio_url=ep["enclosure_url"],
        podcast_title=podcast["title"],
        episode_title=ep["title"],
    )
    print(f"  Job {job.id}: {job.status}")

    # Poll until complete
    while True:
        status = client.jobs.get(job.id)
        if status.status in ("completed", "failed"):
            break
        time.sleep(5)

    if status.status == "completed":
        # Fetch the transcript
        transcript = client.podcasts.get_transcript(status.transcript_id)
        full_text = " ".join(seg["text"] for seg in transcript.segments)
        print(f"  Done: {len(transcript.segments)} segments, {len(full_text)} chars")
    else:
        print(f"  Failed: {status.error}")

Resolving Apple and Spotify links

One thing that saved me hours: Trawl resolves Apple Podcasts and Spotify links to their underlying RSS feeds. When a colleague sends you https://podcasts.apple.com/us/podcast/..., you don't need to manually find the RSS URL.

curl -X POST https://api.gettrawl.com/api/podcasts/resolve \
  -H "Content-Type: application/json" \
  -d '{"url": "https://podcasts.apple.com/us/podcast/all-in-with-chamath/id1502871393"}'

What I learned

The RSS transcript shortcut was the biggest win. When it exists, you get the transcript instantly with zero cost. For my 50-episode batch, 3 episodes had RSS transcripts — those returned in milliseconds while the rest queued for AI transcription.

The async job model matters for batch work. I submit all 50 jobs upfront, then poll for completion in parallel rather than waiting for each one sequentially. The whole batch finishes in about 10 minutes instead of 2+ hours.

Episodes with existing RSS transcripts don't count toward your usage quota — they're always free.

Framework Integrations

If you're feeding podcast transcripts into an AI pipeline, Trawl has native integrations for the two major frameworks.

LangChain — The langchain-trawl partner package gives you a TrawlLoader that accepts podcast URLs (or any Trawl-supported URL) and returns LangChain Document objects:

from langchain_trawl import TrawlLoader

loader = TrawlLoader(urls=["https://podcasts.apple.com/us/podcast/..."])
docs = loader.load()  # → List[Document] with transcript text + metadata

LlamaIndex — Use the TrawlReader to pull podcast transcripts directly into a LlamaIndex ingestion pipeline:

from llama_index.readers.trawl import TrawlReader

reader = TrawlReader(api_key="trawl_your_key")
documents = reader.load_data(urls=["https://podcasts.apple.com/us/podcast/..."])

Both integrations handle URL resolution, transcription polling, and segment concatenation — you get clean documents ready for chunking and embedding.

TwitterLinkedIn