Overview

Flatiron Software built a multi-stage AI pipeline on Claude that ingests up to 60 news sources daily, summarizes each article, consolidates the results by section and delivers a formatted briefing to a distribution list. The client is Cumulus Media, one of the largest audio media companies in the US.

The system pairs two Claude models through Amazon Bedrock: Claude Sonnet for high-volume per-article summarization and Claude Opus for cross-article consolidation. It runs entirely inside the client's AWS account, with a human review step before distribution.
‍

The technical challenge

The goal was a pipeline that turns a large, inconsistent set of inputs into a short, reliable output every day.

  • Heterogeneous inputs. 40 to 60 sources delivered content through RSS, HTML and APIs, with different structures, lengths and failure modes.
  • Accuracy at volume. Dozens of articles per run had to be summarized without invented facts and within strict length limits.
  • Cross-article reasoning. Overlapping stories from different sources had to be merged into one coherent digest per section.
  • Data residency. All content and model calls had to stay inside the client's AWS account.
  • Reliable delivery. Output had to be generated and sent to many recipients without performance degradation, with full error tracking.
    ‍

Why Claude

Flatiron chose Claude because the task demanded factual accuracy at volume first and editorial judgment second, and Claude's model family covers both.

  • Claude Sonnet for per-article summaries. Each run processes dozens of articles. Sonnet gives the speed and cost profile needed for that volume while holding summaries to under 150 words and to the facts in the source.
  • Claude Opus for section consolidation. Merging many summaries into one coherent digest per section requires grouping related stories, removing overlap and keeping a consistent editorial voice. Flatiron assigned this step to Opus, Claude's most capable model.
  • Reliable instruction following. Each section type (Top Stories, Good News, Nerd News) has its own scoped prompt for tone and focus. Claude follows these constraints consistently from run to run.
  • Enterprise-ready deployment. Through Amazon Bedrock, Claude runs inside Cumulus' AWS account, which met the client's data control requirements without a separate vendor integration.
    ‍

Architecture

The platform is a four-stage pipeline. Claude handles the two language tasks, and AWS services handle ingestion, storage and delivery.

  1. Ingestion. A modular crawler on Amazon ECS Fargate uses per-source adapters for RSS, HTML parsing and APIs. It manages rate limits and retries, logs failures to Aurora PostgreSQL, and reaches over 90% source coverage per run.
  2. Summarization with Claude Sonnet. Each article is sent to Claude Sonnet through Amazon Bedrock with a prompt scoped to its section type. Outputs are checked for length (under 150 words) and tone before they are stored.
  3. Consolidation with Claude Opus. Stored summaries for each section are passed to Claude Opus, which clusters related stories, removes overlap and writes one digest per section. Splitting the work keeps high-volume processing on the faster model and reserves Opus for reasoning across articles.
  4. Review and delivery. Drafts appear in a curation interface for final review. On approval, the system generates a master Word document and per-recipient copies in Amazon S3, then sends them asynchronously through Amazon SES with per-recipient status tracking and automatic retries.

The AI layer is an independent module orchestrated through Bedrock. Models can be swapped or added without changes to the rest of the system, and new sources or sections plug in as new adapters and prompts.


Execution insights

Each technical challenge in the project had a direct solution, most of them built around how Claude was prompted and used.

  • Factual, concise summaries. Articles varied widely in type and length. Flatiron used Claude Sonnet with tightly scoped prompts per section type and validated length and tone before storage.
  • One readable digest per section. Many stories overlapped. A second pass with Claude Opus clusters, deduplicates and writes each section in a consistent voice.
  • Editorial control. Editors needed the final say on what goes on air. A curation interface lets the team review and edit Claude's draft before distribution.
  • Clean input from varied sources. Data quality differed across 40 to 60 feeds (RSS, HTML, APIs). Per-source adapters, retry logic and structured error logging make sure Claude receives clean content.
  • Code-free admin. Resends, recipient management and error checks run through a secure REST API with API key and IP allowlist authentication.
    ‍

Results

  • Over 90% source coverage per run across up to 60 heterogeneous sources.
  • Consolidated briefing delivered by 11 PM EST every night.
  • All content and model inference kept inside the client's AWS account.
  • Admin API for recipients, delivery history, resends and error inspection.
  • Modular design ready for new sources, sections, output formats and additional Claude use cases.

    ‍

The project shows a practical pattern for production Claude systems: split the work by model capability, validate output before it moves downstream, and keep a human review step where accuracy matters.