mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-06-11 09:55:59 +00:00
cd5bedaa74
* docs(spec): MiniMax integration for generation skills + new music skill Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(plan): MiniMax generation providers implementation plan Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(skills): add importlib loader + FakeResp for skill tests * test(skills): register loaded module in sys.modules; raise requests.HTTPError in FakeResp * feat(image-generation): add MiniMax provider with env auto-detect Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(image-generation): guard unknown provider, derive ref MIME, strengthen tests Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(video-generation): add MiniMax provider with async poll/download Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(video-generation): surface base_resp errors while polling; add timeout test * feat(podcast-generation): add MiniMax t2a_v2 provider with env auto-detect Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(podcast-generation): restore TTS credential guard; add volcengine + voice tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(music-generation): new MiniMax music skill via skill-creator Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(music-generation): treat empty lyrics as absent; test no-audio-data path * refactor(skills): add request timeouts to MiniMax network calls Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Potential fix for pull request finding 'Explicit returns mixed with implicit (fall through) returns' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * fix(models): strip inconsistent user-message names for MiniMax chat DeerFlow middlewares tag user messages with provenance names (user-input, summary, loop_warning); langchain serializes them into the OpenAI-compatible payload and MiniMax rejects mismatched user-message names with "user name must be consistent (2013)". PatchedChatMiniMax now drops the per-message name from user-role messages. Point the config.example MiniMax models at PatchedChatMiniMax so they also get reasoning_content mapping. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(image-generation): MiniMax sends JSON prompt field, guard 1500-char limit MiniMax image-01 takes one text string capped at 1500 chars, but the skill was sending the whole structured JSON. The MiniMax provider now extracts the JSON `prompt` field (relying on prompt_optimizer to expand it) and fails fast with a clear error before calling the API when that field exceeds 1500 chars. Authoring stays provider-agnostic; Gemini still receives the full JSON. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(podcast-generation): per-provider TTS concurrency and retry/backoff Each TTS provider owns its concurrency internally — MiniMax runs single-threaded to reduce rate-limit failures, Volcengine keeps 4 workers — with automatic retry and backoff on transient HTTP and base_resp errors. No caller-facing concurrency knob. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(skills): address Copilot review comments on generation skills - video: add raise_for_status + timeout to the Gemini download/POST/poll calls so non-2xx responses surface as clear HTTP errors instead of JSON/KeyError or hangs - video: check the task Fail status before the generic base_resp check so the failure keeps its task_id context - video/image: create the output file parent directory before writing (matching music-generation) so nested output paths do not raise FileNotFoundError - music: require a non-empty prompt and fail fast with ValueError instead of sending an empty prompt to the API Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(scripts): reclaim dev ports across worktrees in make stop/dev All deer-flow worktrees (main checkout + linked worktrees) hardcode the same dev ports (8001/3000/2026), so a service started from any worktree must be reclaimable from another. stop_all now resolves the set of worktree roots (DEERFLOW_ROOTS) and treats a process as deer-flow-owned when its open files live under any of them. It also force-kills survivors on 2026 alongside 8001/3000, fixing `make dev` aborting on the nginx port preflight when a prior nginx lingered on 2026. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(view-image): hide the injected image-context message from the UI ViewImageMiddleware injects a HumanMessage (text + base64 images) so the vision model can see viewed images, but it was the only internal injector that set neither hide_from_ui nor a hidden name, so it leaked into the chat UI (and IM channels) as a user bubble reading "Here are the images you've viewed:". Mark it with additional_kwargs={"hide_from_ui": True}, matching todo/dynamic_context injections, which the frontend isHiddenFromUIMessage and the channel sender already honor. The model still receives the full content. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minimax): mark M2.7 models as text-only (no vision) MiniMax M2.7 / M2.7-highspeed do not support vision; only M3 does. The provider config asserted vision support for M2.7 in four places. - config.example.yaml: 4 M2.7 entries -> supports_vision: false - backend/docs/CONFIGURATION.md: M2.7 + highspeed -> supports_vision: false - wizard: add LLMProvider.model_vision_overrides + extra_config_for() so selecting an M2.7 model writes supports_vision: false while M3 (default) keeps vision; wire it through setup_wizard.py - tests: M2.7-highspeed fixture -> supports_vision=False; add test_minimax_vision_is_per_model Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
204 lines
8.1 KiB
Markdown
204 lines
8.1 KiB
Markdown
---
|
|
name: podcast-generation
|
|
description: Use this skill when the user requests to generate, create, or produce podcasts from text content. Converts written content into a two-host conversational podcast audio format with natural dialogue.
|
|
---
|
|
|
|
# Podcast Generation Skill
|
|
|
|
## Overview
|
|
|
|
This skill generates high-quality podcast audio from text content. The workflow includes creating a structured JSON script (conversational dialogue) and executing audio generation through text-to-speech synthesis.
|
|
|
|
## Core Capabilities
|
|
|
|
- Convert any text content (articles, reports, documentation) into podcast scripts
|
|
- Generate natural two-host conversational dialogue (male and female hosts)
|
|
- Synthesize speech audio using text-to-speech
|
|
- Mix audio chunks into a final podcast MP3 file
|
|
- Support both English and Chinese content
|
|
|
|
## Workflow
|
|
|
|
### Step 1: Understand Requirements
|
|
|
|
When a user requests podcast generation, identify:
|
|
|
|
- Source content: The text/article/report to convert into a podcast
|
|
- Language: English or Chinese (based on content)
|
|
- Output location: Where to save the generated podcast
|
|
- You don't need to check the folder under `/mnt/user-data`
|
|
|
|
### Step 2: Create Structured Script JSON
|
|
|
|
Generate a structured JSON script file in `/mnt/user-data/workspace/` with naming pattern: `{descriptive-name}-script.json`
|
|
|
|
The JSON structure:
|
|
```json
|
|
{
|
|
"locale": "en",
|
|
"lines": [
|
|
{"speaker": "male", "paragraph": "dialogue text"},
|
|
{"speaker": "female", "paragraph": "dialogue text"}
|
|
]
|
|
}
|
|
```
|
|
|
|
### Step 3: Execute Generation
|
|
|
|
Call the Python script:
|
|
```bash
|
|
python /mnt/skills/public/podcast-generation/scripts/generate.py \
|
|
--script-file /mnt/user-data/workspace/script-file.json \
|
|
--output-file /mnt/user-data/outputs/generated-podcast.mp3 \
|
|
--transcript-file /mnt/user-data/outputs/generated-podcast-transcript.md
|
|
```
|
|
|
|
Parameters:
|
|
|
|
- `--script-file`: Absolute path to JSON script file (required)
|
|
- `--output-file`: Absolute path to output MP3 file (required)
|
|
- `--transcript-file`: Absolute path to output transcript markdown file (optional, but recommended)
|
|
|
|
> [!IMPORTANT]
|
|
> - Execute the script in one complete call. Do NOT split the workflow into separate steps.
|
|
> - The script handles all TTS API calls and audio generation internally.
|
|
> - Do NOT read the Python file, just call it with the parameters.
|
|
> - Always include `--transcript-file` to generate a readable transcript for the user.
|
|
> - The TTS provider and its concurrency are selected automatically from environment variables — you do not choose or tune them.
|
|
|
|
## Script JSON Format
|
|
|
|
The script JSON file must follow this structure:
|
|
|
|
```json
|
|
{
|
|
"title": "The History of Artificial Intelligence",
|
|
"locale": "en",
|
|
"lines": [
|
|
{"speaker": "male", "paragraph": "Hello Deer! Welcome back to another episode."},
|
|
{"speaker": "female", "paragraph": "Hey everyone! Today we have an exciting topic to discuss."},
|
|
{"speaker": "male", "paragraph": "That's right! We're going to talk about..."}
|
|
]
|
|
}
|
|
```
|
|
|
|
Fields:
|
|
- `title`: Title of the podcast episode (optional, used as heading in transcript)
|
|
- `locale`: Language code - "en" for English or "zh" for Chinese
|
|
- `lines`: Array of dialogue lines
|
|
- `speaker`: Either "male" or "female"
|
|
- `paragraph`: The dialogue text for this speaker
|
|
|
|
## Script Writing Guidelines
|
|
|
|
When creating the script JSON, follow these guidelines:
|
|
|
|
### Format Requirements
|
|
- Only two hosts: male and female, alternating naturally
|
|
- Target runtime: approximately 10 minutes of dialogue (around 40-60 lines)
|
|
- Start with the male host saying a greeting that includes "Hello Deer"
|
|
|
|
### Tone & Style
|
|
- Natural, conversational dialogue - like two friends chatting
|
|
- Use casual expressions and conversational transitions
|
|
- Avoid overly formal language or academic tone
|
|
- Include reactions, follow-up questions, and natural interjections
|
|
|
|
### Content Guidelines
|
|
- Frequent back-and-forth between hosts
|
|
- Keep sentences short and easy to follow when spoken
|
|
- Plain text only - no markdown formatting in the output
|
|
- Translate technical concepts into accessible language
|
|
- No mathematical formulas, code, or complex notation
|
|
- Make content engaging and accessible for audio-only listeners
|
|
- Exclude meta information like dates, author names, or document structure
|
|
|
|
## Podcast Generation Example
|
|
|
|
User request: "Generate a podcast about the history of artificial intelligence"
|
|
|
|
Step 1: Create script file `/mnt/user-data/workspace/ai-history-script.json`:
|
|
```json
|
|
{
|
|
"title": "The History of Artificial Intelligence",
|
|
"locale": "en",
|
|
"lines": [
|
|
{"speaker": "male", "paragraph": "Hello Deer! Welcome back to another fascinating episode. Today we're diving into something that's literally shaping our future - the history of artificial intelligence."},
|
|
{"speaker": "female", "paragraph": "Oh, I love this topic! You know, AI feels so modern, but it actually has roots going back over seventy years."},
|
|
{"speaker": "male", "paragraph": "Exactly! It all started back in the 1950s. The term artificial intelligence was actually coined by John McCarthy in 1956 at a famous conference at Dartmouth."},
|
|
{"speaker": "female", "paragraph": "Wait, so they were already thinking about machines that could think back then? That's incredible!"},
|
|
{"speaker": "male", "paragraph": "Right? The early pioneers were so optimistic. They thought we'd have human-level AI within a generation."},
|
|
{"speaker": "female", "paragraph": "But things didn't quite work out that way, did they?"},
|
|
{"speaker": "male", "paragraph": "No, not at all. The 1970s brought what's called the first AI winter..."}
|
|
]
|
|
}
|
|
```
|
|
|
|
Step 2: Execute generation:
|
|
```bash
|
|
python /mnt/skills/public/podcast-generation/scripts/generate.py \
|
|
--script-file /mnt/user-data/workspace/ai-history-script.json \
|
|
--output-file /mnt/user-data/outputs/ai-history-podcast.mp3 \
|
|
--transcript-file /mnt/user-data/outputs/ai-history-transcript.md
|
|
```
|
|
|
|
This will generate:
|
|
- `ai-history-podcast.mp3`: The audio podcast file
|
|
- `ai-history-transcript.md`: A readable markdown transcript of the podcast
|
|
|
|
## Specific Templates
|
|
|
|
Read the following template file only when matching the user request.
|
|
|
|
- [Tech Explainer](templates/tech-explainer.md) - For converting technical documentation and tutorials
|
|
|
|
## Output Format
|
|
|
|
The generated podcast follows the "Hello Deer" format:
|
|
- Two hosts: one male, one female
|
|
- Natural conversational dialogue
|
|
- Starts with "Hello Deer" greeting
|
|
- Target duration: approximately 10 minutes
|
|
- Alternating speakers for engaging flow
|
|
|
|
## Output Handling
|
|
|
|
After generation:
|
|
|
|
- Podcasts and transcripts are saved in `/mnt/user-data/outputs/`
|
|
- Share both the podcast MP3 and transcript MD with user using `present_files` tool
|
|
- Provide brief description of the generation result (topic, duration, hosts)
|
|
- Offer to regenerate if adjustments needed
|
|
|
|
## Requirements
|
|
|
|
The following environment variables must be set:
|
|
- For Volcengine: `VOLCENGINE_TTS_APPID` and `VOLCENGINE_TTS_ACCESS_TOKEN`
|
|
- For MiniMax: `MINIMAX_API_KEY`
|
|
- `VOLCENGINE_TTS_CLUSTER`: Volcengine TTS cluster (optional, defaults to "volcano_tts")
|
|
|
|
## Notes
|
|
|
|
- **Always execute the full pipeline in one call** - no need to test individual steps or worry about timeouts
|
|
- The script JSON should match the content language (en or zh)
|
|
- Technical content should be simplified for audio accessibility in the script
|
|
- Complex notations (formulas, code) should be translated to plain language in the script
|
|
- Long content may result in longer podcasts
|
|
|
|
## Providers (Volcengine / MiniMax)
|
|
|
|
Auto-selected by environment variables:
|
|
|
|
- `VOLCENGINE_TTS_APPID` + `VOLCENGINE_TTS_ACCESS_TOKEN` set → Volcengine TTS (default).
|
|
- Only `MINIMAX_API_KEY` set → MiniMax TTS (`/v1/t2a_v2`).
|
|
- Force with `PODCAST_GENERATION_PROVIDER=volcengine|minimax`.
|
|
|
|
MiniMax overrides: `MINIMAX_API_HOST` (default `https://api.minimaxi.com`),
|
|
`MINIMAX_TTS_MODEL` (default `speech-2.6-hd`), `MINIMAX_TTS_VOICE_MALE`
|
|
(default `male-qn-qingse`), `MINIMAX_TTS_VOICE_FEMALE` (default `female-tianmei`).
|
|
|
|
Concurrency is owned by each provider internally — MiniMax runs single-threaded
|
|
to reduce rate-limit failures, Volcengine uses 4 workers. There is no
|
|
caller-facing concurrency knob; transient rate limits are handled by automatic
|
|
retry with backoff.
|