Files
deer-flow/docs/superpowers/plans/2026-06-08-minimax-generation-providers.md
T
DanielWalnut cd5bedaa74 feat: MiniMax provider for image/video/podcast skills + new music-generation skill (#3437)
* docs(spec): MiniMax integration for generation skills + new music skill

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(plan): MiniMax generation providers implementation plan

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(skills): add importlib loader + FakeResp for skill tests

* test(skills): register loaded module in sys.modules; raise requests.HTTPError in FakeResp

* feat(image-generation): add MiniMax provider with env auto-detect

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(image-generation): guard unknown provider, derive ref MIME, strengthen tests

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(video-generation): add MiniMax provider with async poll/download

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(video-generation): surface base_resp errors while polling; add timeout test

* feat(podcast-generation): add MiniMax t2a_v2 provider with env auto-detect

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(podcast-generation): restore TTS credential guard; add volcengine + voice tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(music-generation): new MiniMax music skill via skill-creator

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(music-generation): treat empty lyrics as absent; test no-audio-data path

* refactor(skills): add request timeouts to MiniMax network calls

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Potential fix for pull request finding 'Explicit returns mixed with implicit (fall through) returns'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* fix(models): strip inconsistent user-message names for MiniMax chat

DeerFlow middlewares tag user messages with provenance names (user-input, summary, loop_warning); langchain serializes them into the OpenAI-compatible payload and MiniMax rejects mismatched user-message names with "user name must be consistent (2013)". PatchedChatMiniMax now drops the per-message name from user-role messages. Point the config.example MiniMax models at PatchedChatMiniMax so they also get reasoning_content mapping.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(image-generation): MiniMax sends JSON prompt field, guard 1500-char limit

MiniMax image-01 takes one text string capped at 1500 chars, but the skill was sending the whole structured JSON. The MiniMax provider now extracts the JSON `prompt` field (relying on prompt_optimizer to expand it) and fails fast with a clear error before calling the API when that field exceeds 1500 chars. Authoring stays provider-agnostic; Gemini still receives the full JSON.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(podcast-generation): per-provider TTS concurrency and retry/backoff

Each TTS provider owns its concurrency internally — MiniMax runs single-threaded to reduce rate-limit failures, Volcengine keeps 4 workers — with automatic retry and backoff on transient HTTP and base_resp errors. No caller-facing concurrency knob.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(skills): address Copilot review comments on generation skills

- video: add raise_for_status + timeout to the Gemini download/POST/poll calls so non-2xx responses surface as clear HTTP errors instead of JSON/KeyError or hangs
- video: check the task Fail status before the generic base_resp check so the failure keeps its task_id context
- video/image: create the output file parent directory before writing (matching music-generation) so nested output paths do not raise FileNotFoundError
- music: require a non-empty prompt and fail fast with ValueError instead of sending an empty prompt to the API

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(scripts): reclaim dev ports across worktrees in make stop/dev

All deer-flow worktrees (main checkout + linked worktrees) hardcode the same dev ports (8001/3000/2026), so a service started from any worktree must be reclaimable from another. stop_all now resolves the set of worktree roots (DEERFLOW_ROOTS) and treats a process as deer-flow-owned when its open files live under any of them. It also force-kills survivors on 2026 alongside 8001/3000, fixing `make dev` aborting on the nginx port preflight when a prior nginx lingered on 2026.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(view-image): hide the injected image-context message from the UI

ViewImageMiddleware injects a HumanMessage (text + base64 images) so the vision model can see viewed images, but it was the only internal injector that set neither hide_from_ui nor a hidden name, so it leaked into the chat UI (and IM channels) as a user bubble reading "Here are the images you've viewed:". Mark it with additional_kwargs={"hide_from_ui": True}, matching todo/dynamic_context injections, which the frontend isHiddenFromUIMessage and the channel sender already honor. The model still receives the full content.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(minimax): mark M2.7 models as text-only (no vision)

MiniMax M2.7 / M2.7-highspeed do not support vision; only M3 does. The
provider config asserted vision support for M2.7 in four places.

- config.example.yaml: 4 M2.7 entries -> supports_vision: false
- backend/docs/CONFIGURATION.md: M2.7 + highspeed -> supports_vision: false
- wizard: add LLMProvider.model_vision_overrides + extra_config_for() so
  selecting an M2.7 model writes supports_vision: false while M3 (default)
  keeps vision; wire it through setup_wizard.py
- tests: M2.7-highspeed fixture -> supports_vision=False; add
  test_minimax_vision_is_per_model

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
2026-06-08 22:04:38 +08:00

1547 lines
58 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# MiniMax 接入生成类 Skill 实施计划
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 在 image/video/podcast 三个现有 skill 中按环境变量自动接入 MiniMax 作为可选 provider,并用 skill-creator 新建一个 MiniMax 音乐生成 skill。
**Architecture:** 每个 skill 是 `skills/public/<name>/` 下的自包含脚本(`SKILL.md` + `scripts/generate.py`,纯 `requests`)。沙箱内目录隔离,故 MiniMax 代码在每个脚本内各自内联。`generate.py` 顶层用 `_resolve_provider()` 选 provider`<SKILL>_PROVIDER` 覆盖 > 现有 provider 凭证存在 > `MINIMAX_API_KEY` 回退。测试放仓库根 `tests/skills/`,用 `importlib` 按路径加载脚本并 mock `requests`,不打真实 API。
**Tech Stack:** Python 3 + `requests`;测试用 pytest(通过 `uv run --no-project --with pytest --with requests --with Pillow` 运行);新 skill 用 `skills/public/skill-creator/scripts/init_skill.py` 脚手架。
**测试运行命令(全程统一用这条):**
```bash
uv run --no-project --with pytest --with requests --with Pillow pytest tests/skills/ -v
```
**关键事实(来自 MiniMax 官方文档,已核实):**
- Base URL `https://api.minimaxi.com`Header `Authorization: Bearer $MINIMAX_API_KEY` + `Content-Type: application/json`
- 错误判定:响应体 `base_resp.status_code != 0` 即失败。
- 图像 `POST /v1/image_generation` 同步,`response_format:"base64"``data.image_base64[0]`base64)。参考图放 `subject_reference:[{type:"character",image_file:"data:image/jpeg;base64,..."}]`
- 视频三步:`POST /v1/video_generation``task_id``GET /v1/query/video_generation?task_id``status`(`Success`/`Fail`/...)+`file_id``GET /v1/files/retrieve?file_id``file.download_url`;下载 mp4download_url 无需鉴权)。参考图放 `first_frame_image`data URL)。
- 语音 `POST /v1/t2a_v2` 同步 → `data.audio`**hex**`bytes.fromhex`
- 音乐 `POST /v1/music_generation` 同步 → `data.audio`**hex** → mp3。无歌词非纯音乐时 `lyrics_optimizer:true`;纯音乐 `is_instrumental:true`
- 已核实可用 voice_id`male-qn-qingse``female-tianmei`(官方 t2a 文档示例中出现)。
---
## File Structure
**新建:**
- `tests/skills/skill_loader.py` — 按路径加载某 skill 的 `generate.py` 为模块。
- `tests/skills/test_image_generation.py`
- `tests/skills/test_video_generation.py`
- `tests/skills/test_podcast_generation.py`
- `tests/skills/test_music_generation.py`
- `skills/public/music-generation/SKILL.md`(脚手架后替换)
- `skills/public/music-generation/scripts/generate.py`(脚手架后替换)
**修改:**
- `skills/public/image-generation/scripts/generate.py`(整文件替换)
- `skills/public/image-generation/SKILL.md`(追加 MiniMax 说明段)
- `skills/public/video-generation/scripts/generate.py`(整文件替换)
- `skills/public/video-generation/SKILL.md`(追加 MiniMax 说明段)
- `skills/public/podcast-generation/scripts/generate.py`(整文件替换)
- `skills/public/podcast-generation/SKILL.md`(追加 MiniMax 说明段)
- `frontend/src/app/mock/api/skills/route.ts`(新增 music-generation 条目)
---
## Task 0: 测试加载器
**Files:**
- Create: `tests/skills/skill_loader.py`
- [ ] **Step 1: 写加载器**
`tests/skills/skill_loader.py`:
```python
"""Load a skill's scripts/generate.py as an importable module, by file path.
Skills live in skills/public/<name>/scripts/generate.py and are NOT a package,
so tests load them via importlib. Tests then mock the module's `requests`.
"""
import importlib.util
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[2]
def load(skill_name: str):
"""Return the generate.py module for skills/public/<skill_name>."""
path = REPO_ROOT / "skills" / "public" / skill_name / "scripts" / "generate.py"
mod_name = skill_name.replace("-", "_") + "_generate"
spec = importlib.util.spec_from_file_location(mod_name, path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
class FakeResp:
"""Minimal stand-in for requests.Response."""
def __init__(self, json_data=None, content=b"", status_code=200):
self._json = json_data if json_data is not None else {}
self.content = content
self.status_code = status_code
def raise_for_status(self):
if self.status_code >= 400:
raise Exception(f"HTTP {self.status_code}")
def json(self):
return self._json
```
- [ ] **Step 2: 冒烟验证加载器可加载现有脚本**
Run:
```bash
uv run --no-project --with pytest --with requests --with Pillow python -c "import sys; sys.path.insert(0,'tests/skills'); from skill_loader import load; m=load('image-generation'); print('loaded', hasattr(m,'generate_image'))"
```
Expected: 输出 `loaded True`(注意:此步要求 Task 1 尚未执行也能加载——当前 image generate.py 顶层 `from PIL import Image` 需 Pillow,已在命令里 `--with Pillow`)。
- [ ] **Step 3: Commit**
```bash
git add tests/skills/skill_loader.py
git commit -m "test(skills): add importlib loader + FakeResp for skill tests"
```
---
## Task 1: image-generation 接入 MiniMax
**Files:**
- Modify: `skills/public/image-generation/scripts/generate.py`(整文件替换)
- Modify: `skills/public/image-generation/SKILL.md`
- Test: `tests/skills/test_image_generation.py`
- [ ] **Step 1: 写失败测试**
`tests/skills/test_image_generation.py`:
```python
import base64
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent))
from skill_loader import FakeResp, load # noqa: E402
img = load("image-generation")
@pytest.fixture(autouse=True)
def clean_env(monkeypatch):
for k in ["GEMINI_API_KEY", "MINIMAX_API_KEY", "IMAGE_GENERATION_PROVIDER",
"MINIMAX_API_HOST", "MINIMAX_IMAGE_MODEL"]:
monkeypatch.delenv(k, raising=False)
def test_resolve_prefers_gemini(monkeypatch):
monkeypatch.setenv("GEMINI_API_KEY", "g")
monkeypatch.setenv("MINIMAX_API_KEY", "m")
assert img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini",
bool(__import__("os").getenv("GEMINI_API_KEY"))) == "gemini"
def test_resolve_falls_back_to_minimax(monkeypatch):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
assert img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False) == "minimax"
def test_resolve_override_wins(monkeypatch):
monkeypatch.setenv("GEMINI_API_KEY", "g")
monkeypatch.setenv("IMAGE_GENERATION_PROVIDER", "MiniMax")
assert img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", True) == "minimax"
def test_resolve_errors_when_none(monkeypatch):
with pytest.raises(ValueError):
img._resolve_provider("IMAGE_GENERATION_PROVIDER", "gemini", False)
def test_minimax_builds_payload_and_writes(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
raw = b"PNGBYTES"
captured = {}
def fake_post(url, headers=None, json=None, **kw):
captured["url"] = url
captured["headers"] = headers
captured["json"] = json
return FakeResp({"data": {"image_base64": [base64.b64encode(raw).decode()]},
"base_resp": {"status_code": 0, "status_msg": "success"}})
monkeypatch.setattr(img.requests, "post", fake_post)
out = tmp_path / "o.jpg"
prompt_file = tmp_path / "p.json"
prompt_file.write_text("a red apple", encoding="utf-8")
msg = img.generate_image(str(prompt_file), [], str(out), "16:9")
assert out.read_bytes() == raw
assert captured["url"].endswith("/v1/image_generation")
assert captured["headers"]["Authorization"] == "Bearer m"
assert captured["json"]["model"] == "image-01"
assert captured["json"]["response_format"] == "base64"
assert captured["json"]["aspect_ratio"] == "16:9"
assert "Successfully generated image" in msg
def test_minimax_reference_image_as_data_url(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
captured = {}
def fake_post(url, headers=None, json=None, **kw):
captured["json"] = json
return FakeResp({"data": {"image_base64": [base64.b64encode(b"x").decode()]},
"base_resp": {"status_code": 0}})
monkeypatch.setattr(img.requests, "post", fake_post)
ref = tmp_path / "ref.jpg"
ref.write_bytes(b"\xff\xd8refbytes")
prompt_file = tmp_path / "p.json"
prompt_file.write_text("scene", encoding="utf-8")
img.generate_image(str(prompt_file), [str(ref)], str(tmp_path / "o.jpg"), "1:1")
subj = captured["json"]["subject_reference"]
assert subj[0]["type"] == "character"
assert subj[0]["image_file"].startswith("data:image/jpeg;base64,")
def test_minimax_raises_on_base_resp_error(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
def fake_post(url, headers=None, json=None, **kw):
return FakeResp({"base_resp": {"status_code": 1004, "status_msg": "auth failed"}})
monkeypatch.setattr(img.requests, "post", fake_post)
prompt_file = tmp_path / "p.json"
prompt_file.write_text("x", encoding="utf-8")
with pytest.raises(Exception) as e:
img.generate_image(str(prompt_file), [], str(tmp_path / "o.jpg"), "1:1")
assert "1004" in str(e.value)
```
- [ ] **Step 2: 运行测试确认失败**
Run: `uv run --no-project --with pytest --with requests --with Pillow pytest tests/skills/test_image_generation.py -v`
Expected: FAIL`_resolve_provider` / minimax 行为尚不存在)。
- [ ] **Step 3: 整文件替换 generate.py**
`skills/public/image-generation/scripts/generate.py`:
```python
import base64
import os
import requests
MINIMAX_DEFAULT_HOST = "https://api.minimaxi.com"
def validate_image(image_path: str) -> bool:
"""Validate if an image file can be opened and is not corrupted."""
from PIL import Image # lazy import: keeps module importable without Pillow
try:
with Image.open(image_path) as image:
image.verify()
with Image.open(image_path) as image:
image.load()
return True
except Exception as exc:
print(f"Warning: Image '{image_path}' is invalid or corrupted: {exc}")
return False
def _resolve_provider(override_env: str, existing_provider: str, has_existing_creds: bool) -> str:
"""Pick the generation provider.
1. Explicit <SKILL>_PROVIDER override wins.
2. Otherwise prefer the existing provider when its credentials are present.
3. Otherwise fall back to MiniMax when MINIMAX_API_KEY is set.
"""
override = os.getenv(override_env)
if override:
return override.strip().lower()
if has_existing_creds:
return existing_provider
if os.getenv("MINIMAX_API_KEY"):
return "minimax"
raise ValueError(
f"No credentials found. Set GEMINI_API_KEY for {existing_provider}, "
f"or MINIMAX_API_KEY for minimax (optionally force with {override_env})."
)
def _minimax_host() -> str:
return os.getenv("MINIMAX_API_HOST", MINIMAX_DEFAULT_HOST).rstrip("/")
def _check_base_resp(payload: dict) -> None:
base = payload.get("base_resp") or {}
if base.get("status_code", 0) != 0:
raise Exception(
f"MiniMax error {base.get('status_code')}: {base.get('status_msg')}"
)
def _to_data_url(image_path: str) -> str:
with open(image_path, "rb") as f:
b64 = base64.b64encode(f.read()).decode("utf-8")
return f"data:image/jpeg;base64,{b64}"
def _generate_image_minimax(
prompt: str, reference_images: list[str], output_file: str, aspect_ratio: str
) -> str:
api_key = os.getenv("MINIMAX_API_KEY")
if not api_key:
return "MINIMAX_API_KEY is not set"
body = {
"model": os.getenv("MINIMAX_IMAGE_MODEL", "image-01"),
"prompt": prompt,
"aspect_ratio": aspect_ratio,
"response_format": "base64",
"n": 1,
"prompt_optimizer": True,
}
if reference_images:
body["subject_reference"] = [
{"type": "character", "image_file": _to_data_url(p)} for p in reference_images
]
response = requests.post(
f"{_minimax_host()}/v1/image_generation",
headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
json=body,
)
response.raise_for_status()
payload = response.json()
_check_base_resp(payload)
images = (payload.get("data") or {}).get("image_base64") or []
if not images:
raise Exception("MiniMax returned no image data")
with open(output_file, "wb") as f:
f.write(base64.b64decode(images[0]))
return f"Successfully generated image to {output_file}"
def _generate_image_gemini(
prompt: str, reference_images: list[str], output_file: str, aspect_ratio: str
) -> str:
parts = []
valid_reference_images = []
for ref_img in reference_images:
if validate_image(ref_img):
valid_reference_images.append(ref_img)
else:
print(f"Skipping invalid reference image: {ref_img}")
if len(valid_reference_images) < len(reference_images):
skipped = len(reference_images) - len(valid_reference_images)
print(f"Note: {skipped} reference image(s) were skipped due to validation failure.")
for reference_image in valid_reference_images:
with open(reference_image, "rb") as f:
image_b64 = base64.b64encode(f.read()).decode("utf-8")
parts.append({"inlineData": {"mimeType": "image/jpeg", "data": image_b64}})
api_key = os.getenv("GEMINI_API_KEY")
if not api_key:
return "GEMINI_API_KEY is not set"
response = requests.post(
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3-pro-image-preview:generateContent",
headers={"x-goog-api-key": api_key, "Content-Type": "application/json"},
json={
"generationConfig": {"imageConfig": {"aspectRatio": aspect_ratio}},
"contents": [{"parts": [*parts, {"text": prompt}]}],
},
)
response.raise_for_status()
data = response.json()
response_parts: list[dict] = data["candidates"][0]["content"]["parts"]
image_parts = [part for part in response_parts if part.get("inlineData", False)]
if len(image_parts) == 1:
base64_image = image_parts[0]["inlineData"]["data"]
with open(output_file, "wb") as f:
f.write(base64.b64decode(base64_image))
return f"Successfully generated image to {output_file}"
raise Exception("Failed to generate image")
def generate_image(
prompt_file: str,
reference_images: list[str],
output_file: str,
aspect_ratio: str = "16:9",
) -> str:
with open(prompt_file, "r", encoding="utf-8") as f:
prompt = f.read()
provider = _resolve_provider(
"IMAGE_GENERATION_PROVIDER", "gemini", bool(os.getenv("GEMINI_API_KEY"))
)
if provider == "minimax":
return _generate_image_minimax(prompt, reference_images, output_file, aspect_ratio)
return _generate_image_gemini(prompt, reference_images, output_file, aspect_ratio)
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description="Generate images using Gemini or MiniMax API")
parser.add_argument("--prompt-file", required=True, help="Absolute path to JSON prompt file")
parser.add_argument("--reference-images", nargs="*", default=[],
help="Absolute paths to reference images (space-separated)")
parser.add_argument("--output-file", required=True, help="Output path for generated image")
parser.add_argument("--aspect-ratio", required=False, default="16:9",
help="Aspect ratio of the generated image")
args = parser.parse_args()
try:
print(generate_image(args.prompt_file, args.reference_images,
args.output_file, args.aspect_ratio))
except Exception as e:
print(f"Error while generating image: {e}")
```
- [ ] **Step 4: 运行测试确认通过**
Run: `uv run --no-project --with pytest --with requests --with Pillow pytest tests/skills/test_image_generation.py -v`
Expected: PASS7 个用例全过)。
- [ ] **Step 5: 更新 SKILL.md(追加 provider 说明)**
`skills/public/image-generation/SKILL.md``## Notes` 段之前插入新段落:
```markdown
## Providers (Gemini / MiniMax)
This skill auto-selects the provider by environment variables (no CLI change):
- `GEMINI_API_KEY` set → use Gemini (default, unchanged).
- Only `MINIMAX_API_KEY` set → use MiniMax (`/v1/image_generation`, model `image-01`).
- Force one explicitly with `IMAGE_GENERATION_PROVIDER=gemini|minimax`.
MiniMax optional overrides: `MINIMAX_API_HOST` (default `https://api.minimaxi.com`),
`MINIMAX_IMAGE_MODEL` (default `image-01`). Reference images are sent as the MiniMax
`subject_reference` character image. The CLI and `--prompt-file` / `--reference-images`
/ `--output-file` / `--aspect-ratio` arguments are identical for both providers.
```
- [ ] **Step 6: Commit**
```bash
git add skills/public/image-generation/scripts/generate.py skills/public/image-generation/SKILL.md tests/skills/test_image_generation.py
git commit -m "feat(image-generation): add MiniMax provider with env auto-detect"
```
---
## Task 2: video-generation 接入 MiniMax
**Files:**
- Modify: `skills/public/video-generation/scripts/generate.py`(整文件替换)
- Modify: `skills/public/video-generation/SKILL.md`
- Test: `tests/skills/test_video_generation.py`
- [ ] **Step 1: 写失败测试**
`tests/skills/test_video_generation.py`:
```python
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent))
from skill_loader import FakeResp, load # noqa: E402
vid = load("video-generation")
@pytest.fixture(autouse=True)
def clean_env(monkeypatch):
for k in ["GEMINI_API_KEY", "MINIMAX_API_KEY", "VIDEO_GENERATION_PROVIDER",
"MINIMAX_API_HOST", "MINIMAX_VIDEO_MODEL"]:
monkeypatch.delenv(k, raising=False)
monkeypatch.setattr(vid.time, "sleep", lambda *_: None)
def test_resolve_prefers_gemini():
assert vid._resolve_provider("VIDEO_GENERATION_PROVIDER", "gemini", True) == "gemini"
def test_resolve_falls_back_to_minimax(monkeypatch):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
assert vid._resolve_provider("VIDEO_GENERATION_PROVIDER", "gemini", False) == "minimax"
def test_resolve_override(monkeypatch):
monkeypatch.setenv("VIDEO_GENERATION_PROVIDER", "minimax")
assert vid._resolve_provider("VIDEO_GENERATION_PROVIDER", "gemini", True) == "minimax"
def test_minimax_full_flow(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
posts = {}
def fake_post(url, headers=None, json=None, **kw):
posts["url"] = url
posts["json"] = json
return FakeResp({"task_id": "T1", "base_resp": {"status_code": 0}})
def fake_get(url, headers=None, params=None, **kw):
if url.endswith("/v1/query/video_generation"):
assert params["task_id"] == "T1"
return FakeResp({"status": "Success", "file_id": "F1",
"base_resp": {"status_code": 0}})
if url.endswith("/v1/files/retrieve"):
assert params["file_id"] == "F1"
return FakeResp({"file": {"download_url": "https://dl/v.mp4"},
"base_resp": {"status_code": 0}})
return FakeResp(content=b"MP4DATA") # the actual download
monkeypatch.setattr(vid.requests, "post", fake_post)
monkeypatch.setattr(vid.requests, "get", fake_get)
out = tmp_path / "v.mp4"
pf = tmp_path / "p.json"
pf.write_text("a cat runs", encoding="utf-8")
msg = vid.generate_video(str(pf), [], str(out), "16:9")
assert out.read_bytes() == b"MP4DATA"
assert posts["url"].endswith("/v1/video_generation")
assert posts["json"]["model"] == "MiniMax-Hailuo-2.3"
assert "successfully" in msg.lower()
def test_minimax_reference_first_frame(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
posts = {}
def fake_post(url, headers=None, json=None, **kw):
posts["json"] = json
return FakeResp({"task_id": "T1", "base_resp": {"status_code": 0}})
def fake_get(url, headers=None, params=None, **kw):
if url.endswith("/v1/query/video_generation"):
return FakeResp({"status": "Success", "file_id": "F1", "base_resp": {"status_code": 0}})
if url.endswith("/v1/files/retrieve"):
return FakeResp({"file": {"download_url": "https://dl/v.mp4"}, "base_resp": {"status_code": 0}})
return FakeResp(content=b"X")
monkeypatch.setattr(vid.requests, "post", fake_post)
monkeypatch.setattr(vid.requests, "get", fake_get)
ref = tmp_path / "f.jpg"
ref.write_bytes(b"\xff\xd8img")
pf = tmp_path / "p.json"
pf.write_text("x", encoding="utf-8")
vid.generate_video(str(pf), [str(ref)], str(tmp_path / "v.mp4"), "16:9")
assert posts["json"]["first_frame_image"].startswith("data:image/jpeg;base64,")
def test_minimax_task_fail(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
def fake_post(url, headers=None, json=None, **kw):
return FakeResp({"task_id": "T1", "base_resp": {"status_code": 0}})
def fake_get(url, headers=None, params=None, **kw):
return FakeResp({"status": "Fail", "base_resp": {"status_code": 1027, "status_msg": "blocked"}})
monkeypatch.setattr(vid.requests, "post", fake_post)
monkeypatch.setattr(vid.requests, "get", fake_get)
pf = tmp_path / "p.json"
pf.write_text("x", encoding="utf-8")
with pytest.raises(Exception):
vid.generate_video(str(pf), [], str(tmp_path / "v.mp4"), "16:9")
```
- [ ] **Step 2: 运行测试确认失败**
Run: `uv run --no-project --with pytest --with requests --with Pillow pytest tests/skills/test_video_generation.py -v`
Expected: FAIL。
- [ ] **Step 3: 整文件替换 generate.py**
`skills/public/video-generation/scripts/generate.py`:
```python
import base64
import os
import time
import requests
MINIMAX_DEFAULT_HOST = "https://api.minimaxi.com"
def _resolve_provider(override_env: str, existing_provider: str, has_existing_creds: bool) -> str:
"""Pick the provider: <SKILL>_PROVIDER override > existing creds > MiniMax fallback."""
override = os.getenv(override_env)
if override:
return override.strip().lower()
if has_existing_creds:
return existing_provider
if os.getenv("MINIMAX_API_KEY"):
return "minimax"
raise ValueError(
f"No credentials found. Set GEMINI_API_KEY for {existing_provider}, "
f"or MINIMAX_API_KEY for minimax (optionally force with {override_env})."
)
def _minimax_host() -> str:
return os.getenv("MINIMAX_API_HOST", MINIMAX_DEFAULT_HOST).rstrip("/")
def _check_base_resp(payload: dict) -> None:
base = payload.get("base_resp") or {}
if base.get("status_code", 0) != 0:
raise Exception(f"MiniMax error {base.get('status_code')}: {base.get('status_msg')}")
def _to_data_url(image_path: str) -> str:
with open(image_path, "rb") as f:
b64 = base64.b64encode(f.read()).decode("utf-8")
return f"data:image/jpeg;base64,{b64}"
def _poll_video_task(host: str, auth: str, task_id: str,
max_attempts: int = 120, interval: int = 3) -> str:
for _ in range(max_attempts):
response = requests.get(
f"{host}/v1/query/video_generation",
headers={"Authorization": auth},
params={"task_id": task_id},
)
response.raise_for_status()
payload = response.json()
status = payload.get("status")
if status == "Success":
return payload["file_id"]
if status == "Fail":
base = payload.get("base_resp") or {}
raise Exception(
f"MiniMax video task {task_id} failed: "
f"{base.get('status_code')} {base.get('status_msg')}"
)
time.sleep(interval)
raise Exception(f"MiniMax video task {task_id} timed out after {max_attempts} polls")
def _retrieve_file_url(host: str, auth: str, file_id: str) -> str:
response = requests.get(
f"{host}/v1/files/retrieve",
headers={"Authorization": auth},
params={"file_id": file_id},
)
response.raise_for_status()
payload = response.json()
_check_base_resp(payload)
return payload["file"]["download_url"]
def _download(url: str, output_file: str) -> None:
response = requests.get(url)
response.raise_for_status()
with open(output_file, "wb") as f:
f.write(response.content)
def _generate_video_minimax(
prompt: str, reference_images: list[str], output_file: str
) -> str:
api_key = os.getenv("MINIMAX_API_KEY")
if not api_key:
return "MINIMAX_API_KEY is not set"
host = _minimax_host()
auth = f"Bearer {api_key}"
body = {"model": os.getenv("MINIMAX_VIDEO_MODEL", "MiniMax-Hailuo-2.3"), "prompt": prompt}
if reference_images:
body["first_frame_image"] = _to_data_url(reference_images[0])
response = requests.post(
f"{host}/v1/video_generation",
headers={"Authorization": auth, "Content-Type": "application/json"},
json=body,
)
response.raise_for_status()
payload = response.json()
_check_base_resp(payload)
task_id = payload["task_id"]
file_id = _poll_video_task(host, auth, task_id)
download_url = _retrieve_file_url(host, auth, file_id)
_download(download_url, output_file)
return f"The video has been generated successfully to {output_file}"
def download(url: str, output_file: str):
api_key = os.getenv("GEMINI_API_KEY")
if not api_key:
return "GEMINI_API_KEY is not set"
response = requests.get(url, headers={"x-goog-api-key": api_key})
with open(output_file, "wb") as f:
f.write(response.content)
def _generate_video_gemini(
prompt: str, reference_images: list[str], output_file: str
) -> str:
reference_payload = []
request_json = {"instances": [{"prompt": prompt}]}
for reference_image in reference_images:
with open(reference_image, "rb") as f:
image_b64 = base64.b64encode(f.read()).decode("utf-8")
reference_payload.append(
{"image": {"mimeType": "image/jpeg", "bytesBase64Encoded": image_b64},
"referenceType": "asset"}
)
if reference_payload:
request_json["instances"][0]["referenceImages"] = reference_payload
api_key = os.getenv("GEMINI_API_KEY")
if not api_key:
return "GEMINI_API_KEY is not set"
response = requests.post(
"https://generativelanguage.googleapis.com/v1beta/models/veo-3.1-generate-preview:predictLongRunning",
headers={"x-goog-api-key": api_key, "Content-Type": "application/json"},
json=request_json,
)
data = response.json()
operation_name = data["name"]
while True:
response = requests.get(
f"https://generativelanguage.googleapis.com/v1beta/{operation_name}",
headers={"x-goog-api-key": api_key},
)
data = response.json()
if data.get("done", False):
sample = data["response"]["generateVideoResponse"]["generatedSamples"][0]
download(sample["video"]["uri"], output_file)
break
time.sleep(3)
return f"The video has been generated successfully to {output_file}"
def generate_video(
prompt_file: str,
reference_images: list[str],
output_file: str,
aspect_ratio: str = "16:9",
) -> str:
with open(prompt_file, "r", encoding="utf-8") as f:
prompt = f.read()
provider = _resolve_provider(
"VIDEO_GENERATION_PROVIDER", "gemini", bool(os.getenv("GEMINI_API_KEY"))
)
if provider == "minimax":
# MiniMax video uses resolution/duration, not aspect_ratio; aspect_ratio ignored.
return _generate_video_minimax(prompt, reference_images, output_file)
return _generate_video_gemini(prompt, reference_images, output_file)
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description="Generate videos using Gemini or MiniMax API")
parser.add_argument("--prompt-file", required=True, help="Absolute path to JSON prompt file")
parser.add_argument("--reference-images", nargs="*", default=[],
help="Absolute paths to reference images (space-separated)")
parser.add_argument("--output-file", required=True, help="Output path for generated video")
parser.add_argument("--aspect-ratio", required=False, default="16:9",
help="Aspect ratio of the generated video (Gemini only)")
args = parser.parse_args()
try:
print(generate_video(args.prompt_file, args.reference_images,
args.output_file, args.aspect_ratio))
except Exception as e:
print(f"Error while generating video: {e}")
```
- [ ] **Step 4: 运行测试确认通过**
Run: `uv run --no-project --with pytest --with requests --with Pillow pytest tests/skills/test_video_generation.py -v`
Expected: PASS6 个用例全过)。
- [ ] **Step 5: 更新 SKILL.md**
`skills/public/video-generation/SKILL.md` 末尾追加:
```markdown
## Providers (Gemini / MiniMax)
Auto-selected by environment variables (CLI unchanged):
- `GEMINI_API_KEY` set → Gemini Veo (default, unchanged).
- Only `MINIMAX_API_KEY` set → MiniMax video (`/v1/video_generation`, async 3-step poll/download).
- Force with `VIDEO_GENERATION_PROVIDER=gemini|minimax`.
MiniMax overrides: `MINIMAX_API_HOST` (default `https://api.minimaxi.com`),
`MINIMAX_VIDEO_MODEL` (default `MiniMax-Hailuo-2.3`). The first reference image is used
as MiniMax `first_frame_image`. MiniMax ignores `--aspect-ratio` (it uses resolution/duration).
```
- [ ] **Step 6: Commit**
```bash
git add skills/public/video-generation/scripts/generate.py skills/public/video-generation/SKILL.md tests/skills/test_video_generation.py
git commit -m "feat(video-generation): add MiniMax provider with async poll/download"
```
---
## Task 3: podcast-generation 接入 MiniMax
**Files:**
- Modify: `skills/public/podcast-generation/scripts/generate.py`(整文件替换)
- Modify: `skills/public/podcast-generation/SKILL.md`
- Test: `tests/skills/test_podcast_generation.py`
- [ ] **Step 1: 写失败测试**
`tests/skills/test_podcast_generation.py`:
```python
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent))
from skill_loader import FakeResp, load # noqa: E402
pod = load("podcast-generation")
@pytest.fixture(autouse=True)
def clean_env(monkeypatch):
for k in ["VOLCENGINE_TTS_APPID", "VOLCENGINE_TTS_ACCESS_TOKEN", "VOLCENGINE_TTS_CLUSTER",
"MINIMAX_API_KEY", "PODCAST_GENERATION_PROVIDER", "MINIMAX_API_HOST",
"MINIMAX_TTS_MODEL", "MINIMAX_TTS_VOICE_MALE", "MINIMAX_TTS_VOICE_FEMALE"]:
monkeypatch.delenv(k, raising=False)
def test_resolve_prefers_volcengine(monkeypatch):
monkeypatch.setenv("VOLCENGINE_TTS_APPID", "a")
monkeypatch.setenv("VOLCENGINE_TTS_ACCESS_TOKEN", "t")
assert pod._resolve_tts_provider() == "volcengine"
def test_resolve_falls_back_to_minimax(monkeypatch):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
assert pod._resolve_tts_provider() == "minimax"
def test_resolve_override(monkeypatch):
monkeypatch.setenv("VOLCENGINE_TTS_APPID", "a")
monkeypatch.setenv("VOLCENGINE_TTS_ACCESS_TOKEN", "t")
monkeypatch.setenv("PODCAST_GENERATION_PROVIDER", "minimax")
assert pod._resolve_tts_provider() == "minimax"
def test_minimax_tts_decodes_hex(monkeypatch):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
captured = {}
def fake_post(url, headers=None, json=None, **kw):
captured["url"] = url
captured["json"] = json
return FakeResp({"data": {"audio": b"audiobytes".hex(), "status": 2},
"base_resp": {"status_code": 0}})
monkeypatch.setattr(pod.requests, "post", fake_post)
out = pod.text_to_speech_minimax("hello", "male-qn-qingse")
assert out == b"audiobytes"
assert captured["url"].endswith("/v1/t2a_v2")
assert captured["json"]["voice_setting"]["voice_id"] == "male-qn-qingse"
assert captured["json"]["output_format"] == "hex"
def test_process_line_minimax_voice_mapping(monkeypatch):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
seen = {}
def fake_tts(text, voice_id):
seen["voice_id"] = voice_id
return b"x"
monkeypatch.setattr(pod, "text_to_speech_minimax", fake_tts)
line = pod.ScriptLine(speaker="female", paragraph="hi")
idx, audio = pod._process_line((0, line, 1, "minimax"))
assert audio == b"x"
assert seen["voice_id"] == "female-tianmei"
def test_generate_podcast_minimax_end_to_end(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
def fake_post(url, headers=None, json=None, **kw):
return FakeResp({"data": {"audio": b"chunk".hex(), "status": 2},
"base_resp": {"status_code": 0}})
monkeypatch.setattr(pod.requests, "post", fake_post)
script = tmp_path / "s.json"
script.write_text(
'{"title":"T","locale":"en","lines":[{"speaker":"male","paragraph":"a"},'
'{"speaker":"female","paragraph":"b"}]}',
encoding="utf-8",
)
out = tmp_path / "o.mp3"
msg = pod.generate_podcast(str(script), str(out), None)
assert out.read_bytes() == b"chunkchunk"
assert "Successfully generated podcast" in msg
```
- [ ] **Step 2: 运行测试确认失败**
Run: `uv run --no-project --with pytest --with requests --with Pillow pytest tests/skills/test_podcast_generation.py -v`
Expected: FAIL。
- [ ] **Step 3: 整文件替换 generate.py**
`skills/public/podcast-generation/scripts/generate.py`:
```python
import argparse
import base64
import json
import logging
import os
import uuid
from concurrent.futures import ThreadPoolExecutor, as_completed
from typing import Literal, Optional
import requests
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
MINIMAX_DEFAULT_HOST = "https://api.minimaxi.com"
class ScriptLine:
def __init__(self, speaker: Literal["male", "female"] = "male", paragraph: str = ""):
self.speaker = speaker
self.paragraph = paragraph
class Script:
def __init__(self, locale: Literal["en", "zh"] = "en", lines: Optional[list[ScriptLine]] = None):
self.locale = locale
self.lines = lines or []
@classmethod
def from_dict(cls, data: dict) -> "Script":
script = cls(locale=data.get("locale", "en"))
for line in data.get("lines", []):
script.lines.append(
ScriptLine(speaker=line.get("speaker", "male"),
paragraph=line.get("paragraph", ""))
)
return script
def _resolve_provider(override_env: str, existing_provider: str, has_existing_creds: bool) -> str:
override = os.getenv(override_env)
if override:
return override.strip().lower()
if has_existing_creds:
return existing_provider
if os.getenv("MINIMAX_API_KEY"):
return "minimax"
raise ValueError(
f"No credentials found. Set VOLCENGINE_TTS_APPID + VOLCENGINE_TTS_ACCESS_TOKEN "
f"for {existing_provider}, or MINIMAX_API_KEY for minimax "
f"(optionally force with {override_env})."
)
def _resolve_tts_provider() -> str:
has_volc = bool(
os.getenv("VOLCENGINE_TTS_APPID") and os.getenv("VOLCENGINE_TTS_ACCESS_TOKEN")
)
return _resolve_provider("PODCAST_GENERATION_PROVIDER", "volcengine", has_volc)
def text_to_speech_volcengine(text: str, voice_type: str) -> Optional[bytes]:
"""Convert text to speech using Volcengine TTS (returns base64-decoded mp3 bytes)."""
app_id = os.getenv("VOLCENGINE_TTS_APPID")
access_token = os.getenv("VOLCENGINE_TTS_ACCESS_TOKEN")
cluster = os.getenv("VOLCENGINE_TTS_CLUSTER", "volcano_tts")
url = "https://openspeech.bytedance.com/api/v1/tts"
headers = {"Content-Type": "application/json", "Authorization": f"Bearer;{access_token}"}
payload = {
"app": {"appid": app_id, "token": "access_token", "cluster": cluster},
"user": {"uid": "podcast-generator"},
"audio": {"voice_type": voice_type, "encoding": "mp3", "speed_ratio": 1.2},
"request": {"reqid": str(uuid.uuid4()), "text": text,
"text_type": "plain", "operation": "query"},
}
try:
response = requests.post(url, json=payload, headers=headers)
if response.status_code != 200:
logger.error(f"TTS API error: {response.status_code} - {response.text}")
return None
result = response.json()
if result.get("code") != 3000:
logger.error(f"TTS error: {result.get('message')} (code: {result.get('code')})")
return None
audio_data = result.get("data")
if audio_data:
return base64.b64decode(audio_data)
except Exception as e:
logger.error(f"TTS error: {str(e)}")
return None
def text_to_speech_minimax(text: str, voice_id: str) -> Optional[bytes]:
"""Convert text to speech using MiniMax t2a_v2 (returns hex-decoded mp3 bytes)."""
api_key = os.getenv("MINIMAX_API_KEY")
host = os.getenv("MINIMAX_API_HOST", MINIMAX_DEFAULT_HOST).rstrip("/")
payload = {
"model": os.getenv("MINIMAX_TTS_MODEL", "speech-2.6-hd"),
"text": text,
"voice_setting": {"voice_id": voice_id, "speed": 1.0, "vol": 1.0, "pitch": 0},
"audio_setting": {"sample_rate": 32000, "bitrate": 128000, "format": "mp3", "channel": 1},
"output_format": "hex",
}
try:
response = requests.post(
f"{host}/v1/t2a_v2",
headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
json=payload,
)
if response.status_code != 200:
logger.error(f"MiniMax TTS error: {response.status_code} - {response.text}")
return None
result = response.json()
if (result.get("base_resp") or {}).get("status_code", 0) != 0:
base = result.get("base_resp") or {}
logger.error(f"MiniMax TTS error {base.get('status_code')}: {base.get('status_msg')}")
return None
audio_hex = (result.get("data") or {}).get("audio")
if audio_hex:
return bytes.fromhex(audio_hex)
except Exception as e:
logger.error(f"MiniMax TTS error: {str(e)}")
return None
def _process_line(args: tuple[int, ScriptLine, int, str]) -> tuple[int, Optional[bytes]]:
"""Process a single script line for TTS. Returns (index, audio_bytes)."""
i, line, total, provider = args
logger.info(f"Processing line {i + 1}/{total} ({line.speaker}) via {provider}")
if provider == "minimax":
if line.speaker == "male":
voice = os.getenv("MINIMAX_TTS_VOICE_MALE", "male-qn-qingse")
else:
voice = os.getenv("MINIMAX_TTS_VOICE_FEMALE", "female-tianmei")
audio = text_to_speech_minimax(line.paragraph, voice)
else:
if line.speaker == "male":
voice = "zh_male_yangguangqingnian_moon_bigtts"
else:
voice = "zh_female_sajiaonvyou_moon_bigtts"
audio = text_to_speech_volcengine(line.paragraph, voice)
if not audio:
logger.warning(f"Failed to generate audio for line {i + 1}")
return (i, audio)
def tts_node(script: Script, max_workers: int = 4) -> list[bytes]:
"""Convert script lines to audio chunks using TTS with multi-threading."""
total = len(script.lines)
if total == 0:
raise ValueError("Script contains no lines to process")
provider = _resolve_tts_provider()
logger.info(f"Converting script to audio using {max_workers} workers (provider={provider})...")
tasks = [(i, line, total, provider) for i, line in enumerate(script.lines)]
results: dict[int, Optional[bytes]] = {}
failed_indices: list[int] = []
with ThreadPoolExecutor(max_workers=max_workers) as executor:
futures = {executor.submit(_process_line, task): task[0] for task in tasks}
for future in as_completed(futures):
idx, audio = future.result()
results[idx] = audio
if not audio:
failed_indices.append(idx)
if failed_indices:
logger.warning(
f"Failed to generate audio for {len(failed_indices)}/{total} lines: "
f"line numbers {sorted(i + 1 for i in failed_indices)}"
)
audio_chunks = []
for i in range(total):
audio = results.get(i)
if audio:
audio_chunks.append(audio)
logger.info(f"Generated {len(audio_chunks)}/{total} audio chunks successfully")
if not audio_chunks:
raise ValueError(f"TTS generation failed for all {total} lines.")
return audio_chunks
def mix_audio(audio_chunks: list[bytes]) -> bytes:
"""Combine audio chunks into a single audio file."""
if not audio_chunks:
raise ValueError("No audio chunks to mix - TTS generation may have failed")
output = b"".join(audio_chunks)
if len(output) == 0:
raise ValueError("Mixed audio is empty - TTS generation may have failed")
logger.info(f"Audio mixing complete: {len(output)} bytes")
return output
def generate_markdown(script: Script, title: str = "Podcast Script") -> str:
lines = [f"# {title}", ""]
for line in script.lines:
speaker_name = "**Host (Male)**" if line.speaker == "male" else "**Host (Female)**"
lines.append(f"{speaker_name}: {line.paragraph}")
lines.append("")
return "\n".join(lines)
def generate_podcast(script_file: str, output_file: str,
transcript_file: Optional[str] = None) -> str:
with open(script_file, "r", encoding="utf-8") as f:
script_json = json.load(f)
if "lines" not in script_json:
raise ValueError(
f"Invalid script format: missing 'lines' key. Got keys: {list(script_json.keys())}"
)
script = Script.from_dict(script_json)
logger.info(f"Loaded script with {len(script.lines)} lines")
if transcript_file:
title = script_json.get("title", "Podcast Script")
markdown_content = generate_markdown(script, title)
transcript_dir = os.path.dirname(transcript_file)
if transcript_dir:
os.makedirs(transcript_dir, exist_ok=True)
with open(transcript_file, "w", encoding="utf-8") as f:
f.write(markdown_content)
logger.info(f"Generated transcript to {transcript_file}")
audio_chunks = tts_node(script)
if not audio_chunks:
raise Exception("Failed to generate any audio")
output_audio = mix_audio(audio_chunks)
output_dir = os.path.dirname(output_file)
if output_dir:
os.makedirs(output_dir, exist_ok=True)
with open(output_file, "wb") as f:
f.write(output_audio)
result = f"Successfully generated podcast to {output_file}"
if transcript_file:
result += f" and transcript to {transcript_file}"
return result
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Generate podcast from script JSON file")
parser.add_argument("--script-file", required=True, help="Absolute path to script JSON file")
parser.add_argument("--output-file", required=True, help="Output path for generated podcast MP3")
parser.add_argument("--transcript-file", required=False,
help="Output path for transcript markdown file (optional)")
args = parser.parse_args()
try:
result = generate_podcast(args.script_file, args.output_file, args.transcript_file)
print(result)
except Exception as e:
import traceback
print(f"Error generating podcast: {e}")
traceback.print_exc()
```
- [ ] **Step 4: 运行测试确认通过**
Run: `uv run --no-project --with pytest --with requests --with Pillow pytest tests/skills/test_podcast_generation.py -v`
Expected: PASS6 个用例全过)。
- [ ] **Step 5: 更新 SKILL.md**
`skills/public/podcast-generation/SKILL.md` 末尾追加:
```markdown
## Providers (Volcengine / MiniMax)
Auto-selected by environment variables (CLI unchanged):
- `VOLCENGINE_TTS_APPID` + `VOLCENGINE_TTS_ACCESS_TOKEN` set → Volcengine TTS (default).
- Only `MINIMAX_API_KEY` set → MiniMax TTS (`/v1/t2a_v2`).
- Force with `PODCAST_GENERATION_PROVIDER=volcengine|minimax`.
MiniMax overrides: `MINIMAX_API_HOST` (default `https://api.minimaxi.com`),
`MINIMAX_TTS_MODEL` (default `speech-2.6-hd`), `MINIMAX_TTS_VOICE_MALE`
(default `male-qn-qingse`), `MINIMAX_TTS_VOICE_FEMALE` (default `female-tianmei`).
```
- [ ] **Step 6: Commit**
```bash
git add skills/public/podcast-generation/scripts/generate.py skills/public/podcast-generation/SKILL.md tests/skills/test_podcast_generation.py
git commit -m "feat(podcast-generation): add MiniMax t2a_v2 provider with env auto-detect"
```
---
## Task 4: 新建 music-generation skill(用 skill-creator
**Files:**
- Create: `skills/public/music-generation/SKILL.md`
- Create: `skills/public/music-generation/scripts/generate.py`
- Modify: `frontend/src/app/mock/api/skills/route.ts`
- Test: `tests/skills/test_music_generation.py`
- [ ] **Step 1: 用 skill-creator 脚手架生成骨架**
Run:
```bash
uv run --no-project --with pytest python skills/public/skill-creator/scripts/init_skill.py music-generation --path skills/public
```
Expected: 生成 `skills/public/music-generation/`(含 `SKILL.md` 占位 + `scripts/` + `references/` + `assets/`)。随后删除不需要的目录:
```bash
rm -rf skills/public/music-generation/references skills/public/music-generation/assets
rm -f skills/public/music-generation/scripts/example_script.py
```
(若脚手架生成的示例脚本名不同,删除 `scripts/` 下除将创建的 `generate.py` 外的占位文件。)
- [ ] **Step 2: 写失败测试**
`tests/skills/test_music_generation.py`:
```python
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent))
from skill_loader import FakeResp, load # noqa: E402
mus = load("music-generation")
@pytest.fixture(autouse=True)
def clean_env(monkeypatch):
for k in ["MINIMAX_API_KEY", "MINIMAX_API_HOST", "MINIMAX_MUSIC_MODEL"]:
monkeypatch.delenv(k, raising=False)
def _post_ok(captured):
def fake_post(url, headers=None, json=None, **kw):
captured["url"] = url
captured["headers"] = headers
captured["json"] = json
return FakeResp({"data": {"audio": b"songbytes".hex(), "status": 2},
"base_resp": {"status_code": 0}})
return fake_post
def test_with_lyrics_payload_and_writes(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
captured = {}
monkeypatch.setattr(mus.requests, "post", _post_ok(captured))
spec = tmp_path / "s.json"
spec.write_text('{"title":"X","prompt":"pop, happy","lyrics":"[verse]\\nla la"}',
encoding="utf-8")
out = tmp_path / "o.mp3"
msg = mus.generate_music(str(spec), str(out))
assert out.read_bytes() == b"songbytes"
assert captured["url"].endswith("/v1/music_generation")
assert captured["headers"]["Authorization"] == "Bearer m"
assert captured["json"]["model"] == "music-2.6-free"
assert captured["json"]["lyrics"] == "[verse]\nla la"
assert captured["json"]["output_format"] == "hex"
assert "Successfully generated music" in msg
def test_instrumental_sets_flag(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
captured = {}
monkeypatch.setattr(mus.requests, "post", _post_ok(captured))
spec = tmp_path / "s.json"
spec.write_text('{"prompt":"lofi beats","is_instrumental":true}', encoding="utf-8")
mus.generate_music(str(spec), str(tmp_path / "o.mp3"))
assert captured["json"]["is_instrumental"] is True
assert "lyrics" not in captured["json"]
assert "lyrics_optimizer" not in captured["json"]
def test_no_lyrics_uses_optimizer(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
captured = {}
monkeypatch.setattr(mus.requests, "post", _post_ok(captured))
spec = tmp_path / "s.json"
spec.write_text('{"prompt":"sad ballad"}', encoding="utf-8")
mus.generate_music(str(spec), str(tmp_path / "o.mp3"))
assert captured["json"]["lyrics_optimizer"] is True
assert "lyrics" not in captured["json"]
def test_model_override(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
monkeypatch.setenv("MINIMAX_MUSIC_MODEL", "music-2.6")
captured = {}
monkeypatch.setattr(mus.requests, "post", _post_ok(captured))
spec = tmp_path / "s.json"
spec.write_text('{"prompt":"jazz","lyrics":"[verse]\\nhi"}', encoding="utf-8")
mus.generate_music(str(spec), str(tmp_path / "o.mp3"))
assert captured["json"]["model"] == "music-2.6"
def test_raises_on_base_resp_error(monkeypatch, tmp_path):
monkeypatch.setenv("MINIMAX_API_KEY", "m")
def fake_post(url, headers=None, json=None, **kw):
return FakeResp({"base_resp": {"status_code": 1008, "status_msg": "no balance"}})
monkeypatch.setattr(mus.requests, "post", fake_post)
spec = tmp_path / "s.json"
spec.write_text('{"prompt":"x","lyrics":"[verse]\\ny"}', encoding="utf-8")
with pytest.raises(Exception) as e:
mus.generate_music(str(spec), str(tmp_path / "o.mp3"))
assert "1008" in str(e.value)
def test_missing_api_key_returns_message(monkeypatch, tmp_path):
spec = tmp_path / "s.json"
spec.write_text('{"prompt":"x"}', encoding="utf-8")
msg = mus.generate_music(str(spec), str(tmp_path / "o.mp3"))
assert "MINIMAX_API_KEY" in msg
```
- [ ] **Step 3: 运行测试确认失败**
Run: `uv run --no-project --with pytest --with requests --with Pillow pytest tests/skills/test_music_generation.py -v`
Expected: FAIL`generate_music` 不存在)。
- [ ] **Step 4: 写实现 generate.py**
`skills/public/music-generation/scripts/generate.py`:
```python
import argparse
import json
import os
import requests
MINIMAX_DEFAULT_HOST = "https://api.minimaxi.com"
def _check_base_resp(payload: dict) -> None:
base = payload.get("base_resp") or {}
if base.get("status_code", 0) != 0:
raise Exception(f"MiniMax error {base.get('status_code')}: {base.get('status_msg')}")
def generate_music(prompt_file: str, output_file: str) -> str:
"""Generate a song from a JSON spec via MiniMax /v1/music_generation.
Spec JSON: {"title": str, "prompt": str, "lyrics"?: str, "is_instrumental"?: bool}
- lyrics given -> use them (supports [Verse]/[Chorus] structure tags, \\n lines)
- is_instrumental true -> pure music, no lyrics needed
- otherwise -> lyrics_optimizer auto-writes lyrics from prompt
"""
with open(prompt_file, "r", encoding="utf-8") as f:
spec = json.load(f)
api_key = os.getenv("MINIMAX_API_KEY")
if not api_key:
return "MINIMAX_API_KEY is not set"
prompt = spec.get("prompt", "")
lyrics = spec.get("lyrics")
is_instrumental = bool(spec.get("is_instrumental", False))
body = {
"model": os.getenv("MINIMAX_MUSIC_MODEL", "music-2.6-free"),
"prompt": prompt,
"output_format": "hex",
"audio_setting": {"sample_rate": 44100, "bitrate": 256000, "format": "mp3"},
}
if lyrics:
body["lyrics"] = lyrics
elif is_instrumental:
body["is_instrumental"] = True
else:
body["lyrics_optimizer"] = True
host = os.getenv("MINIMAX_API_HOST", MINIMAX_DEFAULT_HOST).rstrip("/")
response = requests.post(
f"{host}/v1/music_generation",
headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
json=body,
)
response.raise_for_status()
payload = response.json()
_check_base_resp(payload)
audio_hex = (payload.get("data") or {}).get("audio")
if not audio_hex:
raise Exception("MiniMax returned no audio data")
output_dir = os.path.dirname(output_file)
if output_dir:
os.makedirs(output_dir, exist_ok=True)
with open(output_file, "wb") as f:
f.write(bytes.fromhex(audio_hex))
return f"Successfully generated music to {output_file}"
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Generate music using MiniMax API")
parser.add_argument("--prompt-file", required=True,
help="Absolute path to JSON spec file {title, prompt, lyrics?, is_instrumental?}")
parser.add_argument("--output-file", required=True, help="Output path for generated MP3")
args = parser.parse_args()
try:
print(generate_music(args.prompt_file, args.output_file))
except Exception as e:
print(f"Error while generating music: {e}")
```
- [ ] **Step 5: 运行测试确认通过**
Run: `uv run --no-project --with pytest --with requests --with Pillow pytest tests/skills/test_music_generation.py -v`
Expected: PASS6 个用例全过)。
- [ ] **Step 6: 写 SKILL.md**
整文件替换 `skills/public/music-generation/SKILL.md`:
```markdown
---
name: music-generation
description: Use this skill when the user requests to generate, create, compose, or produce music or songs — background music, theme songs, jingles, or instrumental tracks. Generates a song from a style/mood prompt and optional lyrics via the MiniMax music API.
---
# Music Generation Skill
## Overview
This skill generates songs (vocal or instrumental) from a structured JSON spec using the
MiniMax music generation API (`/v1/music_generation`). You describe the style/mood/scene in
`prompt`, optionally provide `lyrics`, and the script returns an MP3.
## Workflow
### Step 1: Understand Requirements
Identify the desired style, mood, scene, language, and whether the user wants vocals or a
pure instrumental track. Decide whether to supply lyrics or let the model write them.
### Step 2: Create the Spec JSON
Write a JSON file in `/mnt/user-data/workspace/` named `{descriptive-name}.json`:
```json
{
"title": "Rainy Night Cafe",
"prompt": "indie folk, melancholic, introspective, walking alone, cafe",
"lyrics": "[verse]\nStreetlights glow the night wind sighs\n[chorus]\nPush the wooden door warm air inside"
}
```
Fields:
- `title` (optional): a human-readable name.
- `prompt` (required): style, mood, and scene. Drives the musical character.
- `lyrics` (optional): song lyrics. Use `\n` between lines and structure tags such as
`[Intro]`, `[Verse]`, `[Pre Chorus]`, `[Chorus]`, `[Bridge]`, `[Outro]`.
- `is_instrumental` (optional, bool): set `true` for a pure instrumental track (no lyrics needed).
Behavior:
- `lyrics` provided → those lyrics are sung.
- `is_instrumental: true` → instrumental, no vocals.
- neither → the model auto-writes lyrics from `prompt` (`lyrics_optimizer`).
### Step 3: Execute Generation
```bash
python /mnt/skills/public/music-generation/scripts/generate.py \
--prompt-file /mnt/user-data/workspace/rainy-night-cafe.json \
--output-file /mnt/user-data/outputs/rainy-night-cafe.mp3
```
Parameters:
- `--prompt-file`: Absolute path to the JSON spec (required).
- `--output-file`: Absolute path for the output MP3 (required).
[!NOTE]
Do NOT read the python file, just call it with the parameters.
## Environment
- `MINIMAX_API_KEY` (required): your MiniMax interface key.
- `MINIMAX_API_HOST` (optional): default `https://api.minimaxi.com`.
- `MINIMAX_MUSIC_MODEL` (optional): default `music-2.6-free` (works for all API-key users);
paid/Token-Plan users can set `music-2.6` for higher limits.
## Output Handling
- Music is saved as MP3 (typically in `/mnt/user-data/outputs/`).
- Share the generated file with the user using the present_files tool.
- Offer to iterate on style or lyrics if adjustments are needed.
## Notes
- Keep `prompt` focused on style/mood/scene; put the actual sung words in `lyrics`.
- For non-English songs, write `lyrics` in the target language.
```
- [ ] **Step 7: 在前端 mock skills 列表注册 music-generation**
修改 `frontend/src/app/mock/api/skills/route.ts`,在 `image-generation` 条目之后、`podcast-generation` 条目之前插入(保持字母序):
```typescript
{
name: "music-generation",
description:
"Use this skill when the user requests to generate, create, compose, or produce music or songs — background music, theme songs, jingles, or instrumental tracks. Generates a song from a style/mood prompt and optional lyrics via the MiniMax music API.",
license: null,
category: "public",
enabled: true,
},
```
- [ ] **Step 8: 前端类型检查(确认 route.ts 无误)**
Run: `cd frontend && pnpm typecheck`
Expected: PASS(无新增类型错误)。若 `frontend` 依赖未安装,先 `pnpm install` 再 typecheck。
- [ ] **Step 9: Commit**
```bash
git add skills/public/music-generation frontend/src/app/mock/api/skills/route.ts tests/skills/test_music_generation.py
git commit -m "feat(music-generation): new MiniMax music skill via skill-creator"
```
---
## Task 5: 全量回归 + spec 覆盖核对
- [ ] **Step 1: 跑全部 skill 测试**
Run: `uv run --no-project --with pytest --with requests --with Pillow pytest tests/skills/ -v`
Expected: 全部 PASSimage 7 + video 6 + podcast 6 + music 6 = 25 用例)。
- [ ] **Step 2: 核对四个 skill 目录结构**
Run:
```bash
ls skills/public/music-generation skills/public/music-generation/scripts
git status --short
```
Expected: `music-generation/SKILL.md` + `scripts/generate.py` 存在;无意外残留的脚手架占位文件(references/assets 已删)。
- [ ] **Step 3: spec 覆盖自查(对照设计文档)**
逐条确认:image/video/podcast 三个 provider 自动判断 + 覆盖 ✔;music 新 skill ✔;hex 解码(podcast+music)✔;base64image)✔;video 三步轮询 ✔;参考图 data URLimage subject_reference / video first_frame_image)✔;前端注册 ✔;环境变量齐全 ✔。如发现遗漏,补任务。
- [ ] **Step 4: 最终提交(如有零散改动)**
```bash
git add -A
git commit -m "test(skills): full MiniMax generation regression green" || echo "nothing to commit"
```