Compare commits

...

102 Commits

Author SHA1 Message Date
Henry Li fa20cc9a98 chore: update mock data 2025-07-23 09:02:04 +08:00
DanielWalnut c7edaf3e84 refine the research prompt (#459) 2025-07-22 14:13:10 +08:00
orifake e6ba1fcd82 fix: JSON parse error in link.tsx (#448)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-07-20 14:14:18 +08:00
Willem Jiang 4d65d20f01 fix: keep applying quick fix for #446 (#450)
* fix: the Backend returns 400 error

* fix: keep applying quick fix

* fix the lint error

* fixed .env.example settings
2025-07-20 14:10:46 +08:00
Willem Jiang ff67366c5c fix: the Backend returns 400 error (#449) 2025-07-20 11:38:18 +08:00
Willem Jiang d34f48819d feat: polish the mcp-server configure feature (#447)
* feat: disable the MCP server configuation by default

* Fixed the lint and test errors

* fix the lint error

* feat:update the mcp config documents and tests

* fixed the lint errors
2025-07-19 09:33:32 +08:00
Willem Jiang 75ad3e0dc6 feat: disable the MCP server configuation by default (#444)
* feat: disable the MCP server configuation by default

* Fixed the lint and test errors

* fix the lint error
2025-07-19 08:39:42 +08:00
DanielWalnut dbb24d7d14 fix: fix the bug introduced by coordinator messages update (#445) 2025-07-18 21:36:13 +08:00
Willem Jiang 933f3bb83a feat: add CORS setting for the backend application (#443)
* feat: add CORS setting for the backend application

* fix the formate issue
2025-07-18 18:04:03 +08:00
道心坚定韩道友 f17b06f206 fix:planner AttributeError 'list' object has no attribute 'get' (#436) 2025-07-18 09:27:15 +08:00
xiaofeng c14c548e0c fix:The console UI directly throws an error when user input is empty (#438) 2025-07-17 18:15:13 +08:00
Kuro Akuta c89b35805d fix: fix the coordinator's forgetting of its own messages. (#433) 2025-07-17 08:36:31 +08:00
DanielWalnut 774473cc18 fix: fix unit test cases for prompt enhancer (#431) 2025-07-16 11:41:43 +08:00
Affan Shaikhsurab b04225b7c8 fix: handle empty agent tuple in streaming workflow (#427)
Prevents IndexError when agent[0] is accessed on empty tuple,
resolving display issues with Gemini 2.0 Flash model.

Fixes #425
2025-07-16 08:59:11 +08:00
DanielWalnut b155e1eca6 refactor: refine the prompt enhancer pipeline (#426) 2025-07-15 19:21:59 +08:00
DanielWalnut 448001f532 refactor: human feedback doesn't need to check enough context (#423) 2025-07-15 18:51:41 +08:00
Willem Jiang 0f118fda92 fix: clean up the builder code (#417)
* fix: clean up the builder code

* fix:reformat the code
2025-07-15 17:22:50 +08:00
Affan Shaikhsurab ae30517f48 Update configuration_guide.md (#416)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-07-14 19:05:07 +08:00
orifake 8bdc6bfa2d fix: missing i18n message (#410)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-07-14 18:56:17 +08:00
HagonChan afbcdd68d8 fix: add missing translation for chat.page (#409)
* fix: Error: MISSING_MESSAGE: Could not resolve chat.page` in messages for locale 'en'

Fixed a `MISSING_MESSAGE` error that was occurring on the chat page due to missing translation keys for `chat.page` in the internationalization messages.

* Update en.json
2025-07-14 18:54:01 +08:00
Willem Jiang bf3bcee8e3 fix: main build fix for the merge #237 (#407) 2025-07-13 09:44:28 +08:00
Shiwen Cheng 0c46f8361b feat: support AzureChatOpenAI under configuring azure_endpoint or AZURE_OPENAI_ENDPOINT (#237)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-07-13 09:27:57 +08:00
Willem Jiang 86a89acac3 fix: update the reasoning model url in conf.yaml.example (#406) 2025-07-13 08:22:37 +08:00
Willem Jiang 2121510f63 fix:catch toolCalls doesn't return validate json (#405)
Co-authored-by: Willem Jiang <143703838+willem-bd@users.noreply.github.com>
2025-07-12 23:31:43 +08:00
cmq2525 0dc6c16c42 fix: repair_json_output cannot process msgs that do not starts with {, [ or ``` (#384)
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-07-12 23:29:22 +08:00
vvky 5abf8c1f5e fix: correctly remove outermost code block markers in model responses (fix markdown rendering issue) (#386)
* fix: correctly remove outermost code block markers in frontend

* fix: correctly remove outermost quote block markers in 'dropMarkdownQuote'

* fix: correctly remove outermost quote block markers in 'dropMarkdownQuote'

* fix: correctly remove outermost quote block markers in 'dropMarkdownQuote'

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-07-12 22:19:30 +08:00
Willem Jiang 70b86d8464 feat: add the Chinese i8n support on the setting table (#404)
* feat: Added i8n to the mcp table

* feat: Added i8n to the about table
2025-07-12 21:28:08 +08:00
johnny0120 e1187d7d02 feat: add i18n support and add Chinese (#372)
* feat: add i18n support and add Chinese

* fix: resolve conflicts

* Update en.json with cancle settings

* Update zh.json with settngs cancle

---------

Co-authored-by: johnny0120 <15564476+johnny0120@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Willem Jiang <143703838+willem-bd@users.noreply.github.com>
2025-07-12 15:18:28 +08:00
Willem Jiang 136f7eaa4e fix:upgrade uv version to avoid the big change of uv.lock (#402)
Co-authored-by: Willem Jiang <143703838+willem-bd@users.noreply.github.com>
2025-07-12 14:46:17 +08:00
Willem Jiang 3c46201ff0 fix: fix the lint check errors of the main branch (#403) 2025-07-12 14:43:25 +08:00
yihong 2363b21447 fix: some lint fix using tools (#98)
* fix: some lint fix using tools

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: md lint

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: some lint fix using tools

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: address comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: tests

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-07-12 13:59:02 +08:00
Willem Jiang 0d3255cdae feat: add the vscode unit test debug settings (#346) 2025-07-12 13:27:47 +08:00
Kirk Lin 9f8f060506 feat(llm): Add retry mechanism for LLM API calls (#400)
* feat(llm): Add retry mechanism for LLM API calls

* feat: configure max_retries for LLM calls via conf.yaml

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-07-12 10:12:07 +08:00
HagonChan dfd4712d9f feat: add Domain Control Features for Tavily Search Engine (#401)
* feat: add Domain Control Features for Tavily Search Engine

* fixed

* chore: update config.md
2025-07-12 08:53:51 +08:00
MaojiaSheng 859c6e3c5d doc: add knowledgebase rag examples in readme (#383)
* doc: add private knowledgebase examples in readme

* doc: add private knowledgebase examples in readme

* doc: add private knowledgebase examples in readme

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-07-07 13:05:01 +08:00
Willem Jiang d8016809b2 fix: the typo of setup-uv action (#393)
* fix: spine the github hash on the third party actions

* fix: the typo of action

* fix: try to fix the build by specify the action version
2025-07-07 08:43:11 +08:00
Willem Jiang 6c254c0783 fix: spine the github hash on the third party actions (#392) 2025-07-07 08:18:17 +08:00
殷逸维 d4fbc86b28 fix: docker build (#385)
* fix: docker build

* modify base docker image
2025-07-05 11:07:52 +08:00
Abeautifulsnow 7ad11bf86c refactor: simplify style mapping by using upper case only (#378)
* improve: add abort btn to abort the mcp add request.

* refactor: simplify style mapping by using upper case only

* format: execute uv run black --preview . to format python files.
2025-07-04 08:27:20 +08:00
殷逸维 be893eae2b feat: integrate VikingDB Knowledge Base into rag retrieving tool (#381)
Co-authored-by: Henry Li <henry1943@163.com>
2025-07-03 10:06:42 +08:00
Johannes Maron 5977b4a03e Publish containers to GitHub (#375)
This workflow creates two offical container images:

* `ghcr.io/codingjoe/deer-flow:main`
* `ghcr.io/codingjoe/deer-flow-web:main`
2025-06-29 20:55:51 +08:00
JeffJiang 52dfdd83ae fix: next server fetch error (#374) 2025-06-27 14:23:04 +08:00
Willem Jiang f27c96e692 fix: the lint error of llm.py (#369) 2025-06-26 10:36:26 +08:00
Tony M b7373fbe70 Add support for self-signed certs from model providers (#276)
* Add support for self-signed certs from model providers

* cleanup

---------

Co-authored-by: tonydoesathing <tmastromarino@cpacket.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2025-06-26 10:17:26 +08:00
Abeautifulsnow 9c2d4724e3 improve: add abort btn to abort the mcp add request. (#284) 2025-06-26 08:51:46 +08:00
johnny0120 aa06cd6fb6 fix: replace json before js fence (#344) 2025-06-26 08:40:32 +08:00
Young 82e1b65792 fix: settings tab display name (#250) 2025-06-19 14:33:00 +08:00
Willem Jiang dcdd7288ed test: add unit tests of the app (#305)
* test: add unit tests in server

* test: add unit tests of app.py in server

* test: reformat the codes

* test: add more tests to cover the exception part

* test: add more tests on the server app part

* fix: don't show the detail exception to the client

* test: try to fix the CI test

* fix: keep the TTS API call without exposure information

* Fixed the unit test errors

* Fixed the lint error
2025-06-18 14:13:05 +08:00
Willem Jiang 89f3d731c9 Fix: the test errors of test_nodes (#345) 2025-06-18 11:59:33 +08:00
Willem Jiang c0b04aaba2 test: add unit tests for graph (#296)
* test: added unit test of builder

* test: Add unit tests for nodes.py

* test: add more unit tests in test_nodes

* test: try to fix the unit test error on GitHub

* test: reformate the code of test_nodes.py

* Fix the test error of reset the local argument

* Fixed the test error by setup args

* reformat the code
2025-06-18 10:05:02 +08:00
Willem Jiang 4048ca67dd test: add test of json_utils (#309)
* test: add test of json_utils

* reformat the code
2025-06-18 10:04:46 +08:00
lelili2021 30a189cf26 fix: update several links related to volcengine in Readme (#333) 2025-06-17 08:49:29 +08:00
Ryan Guo e03b12b97f doc: provide a workable guideline update for ollama user (#323) 2025-06-17 08:47:54 +08:00
Luludle 8823ffdb6a fix: add line breaks to mcp edit dialog (#313) 2025-06-17 08:31:35 +08:00
3Spiders 4fe43153b1 fix(web): priority displayName for settings name error (#336) 2025-06-17 08:26:13 +08:00
Willem Jiang 4fb053b6d2 Revert "fix: solves the malformed json output and pydantic validation error p…" (#325)
This reverts commit a7315b46df.
2025-06-14 22:04:03 +08:00
DanielWalnut 19fa1e97c3 feat: add deep think feature (#311)
* feat: implement backend logic

* feat: implement api/config endpoint

* rename the symbol

* feat: re-implement configuration at client-side

* feat: add client-side of deep thinking

* fix backend bug

* feat: add reasoning block

* docs: update readme

* fix: translate into English

* fix: change icon to lightbulb

* feat: ignore more bad cases

* feat: adjust thinking layout, and implement auto scrolling

* docs: add comments

---------

Co-authored-by: Henry Li <henry1943@163.com>
2025-06-14 13:12:43 +08:00
Tax a7315b46df fix: solves the malformed json output and pydantic validation error produced by the 'planner' node by forcing the llm response to strictly comply with the pydantic 'Plan' model (#322) 2025-06-14 10:13:30 +08:00
JeffJiang 03e6a1a6e7 fix: mcp config styles (#320) 2025-06-13 18:01:19 +08:00
Lan 7d38e5f900 feat: append try catch (#280) 2025-06-12 20:43:50 +08:00
Willem Jiang 4c2fe2e7f5 test: add more unit tests of tools (#315)
* test: add more test on test_tts.py

* test: add unit test of search and retriever in tools

* test: remove the main code of search.py

* test: add the travily_search unit test

* reformate the codes

* test: add unit tests of tools

* Added the pytest-asyncio dependency

* added the license header of test_tavily_search_api_wrapper.py
2025-06-12 20:43:32 +08:00
Henry Li bb7dc6e98c docs: add VolcEngine introduction. (#314) 2025-06-12 13:36:29 +08:00
Willem Jiang ee1af78767 test: added unit tests for rag (#298)
* test: added unit tests for rag

* reformate the code
2025-06-11 19:46:08 +08:00
Willem Jiang 2554e4ba63 test: add unit tests of llms (#299) 2025-06-11 19:46:01 +08:00
LeoJiaXin 397ac57235 fix: input text not clear when click submit button (#303) 2025-06-11 11:11:48 +08:00
DanielWalnut 447e427fd3 refactor: refine teh background check logic (#306) 2025-06-11 11:10:02 +08:00
Muharrem Okutan eeff1ebf80 feat: added report download button (#78) 2025-06-11 09:50:48 +08:00
DanielWalnut 1cd6aa0ece feat: implement enhance prompt (#294)
* feat: implement enhance prompt

* add unit test

* fix prompt

* fix: fix eslint and compiling issues

* feat: add border-beam animation

* fix: fix importing issues

---------

Co-authored-by: Henry Li <henry1943@163.com>
2025-06-08 19:41:59 +08:00
Willem Jiang 8081a14c21 test:unit tests for configuration (#291)
* test:unit tests for configuration

* test: update the test_configuration.py file

* test: reformate the test codes
2025-06-07 21:51:26 +08:00
Willem Jiang c6ed423021 test: add unit tests of crawler (#292)
* test: add unit tests of crawler

* test: polish the code of crawler unit tests
2025-06-07 21:51:05 +08:00
DanielWalnut 0e22c373af feat: support to adjust writing style (#290)
* feat: implment backend for adjust report style

* feat: add web part

* fix test cases

* fix: fix typing

---------

Co-authored-by: Henry Li <henry1943@163.com>
2025-06-07 20:48:39 +08:00
Xintao Wang cda3870add fix: enable proxy support in aiohttp by adding trust_env=True (#289) 2025-06-07 15:30:13 +08:00
DanielWalnut b5ec61bb9d refactor: refine the graph structure (#283) 2025-06-05 12:47:17 +08:00
JeffJiang 73ac8ae45a fix: web start with dotenv (#282) 2025-06-05 11:53:49 +08:00
Xavi 91648c4210 fix: correct placeholder for API key in configuration guide (#278) 2025-06-05 09:46:47 +08:00
Willem Jiang 95257800d2 fix: do not return the server side exception to client (#277) 2025-06-05 09:23:42 +08:00
Willem Jiang 45568ca95b fix:added sanitizing check on the log message (#272)
* fix:added sanitizing check on the log message

* fix: reformat the codes
2025-06-03 11:50:54 +08:00
Willem Jiang db3e74629f fix: added permissions setting in the workflow (#273)
* fix: added permissions setting in the workflow

* fix: reformat the code of src/tools/retriever.py
2025-06-03 11:48:51 +08:00
SToneX 0da52d41a7 feat(chat): add animated deer to response indicator (#269) 2025-05-31 19:13:13 +08:00
Aeolusw eaaad27e44 fix: normalize line endings for consistent chunk splitting (#235) 2025-05-29 20:46:57 +08:00
JeffJiang 4ddd659d8d feat: rag retrieving tool call result display (#263)
* feat: local search tool call result display

* chore: add file copyright

* fix: miss edit plan interrupt feedback

* feat: disable pasting html into input box
2025-05-29 19:52:34 +08:00
JeffJiang 7e9fbed918 fix: editing plan style (#261) 2025-05-29 10:46:05 +08:00
JeffJiang fcbc7f1118 revert: scroll container display change (#258) 2025-05-28 19:23:32 +08:00
JeffJiang d14fb262ea fix: message block width (#257) 2025-05-28 19:11:20 +08:00
JeffJiang 9888098f8a fix: message input box reflow (#252) 2025-05-28 16:38:28 +08:00
DanielWalnut 56e35c6b7f feat: support llm env in env file (#251) 2025-05-28 16:21:40 +08:00
JeffJiang 462752b462 feat: RAG Integration (#238)
* feat: add rag provider and retriever

* feat: retriever tool

* feat: add retriever tool to the researcher node

* feat: add rag http apis

* feat: new message input supports resource mentions

* feat: new message input component support resource mentions

* refactor: need_web_search to need_search

* chore: RAG integration docs

* chore: change example api host

* fix: user message color in dark mode

* fix: mentions style

* feat: add local_search_tool to researcher prompt

* chore: research prompt

* fix: ragflow page size and reporter with

* docs: ragflow integration and add acknowledgment projects

* chore: format
2025-05-28 14:13:46 +08:00
DanielWalnut 0565ab6d27 fix: fix unittes & background investigation search logic (#247) 2025-05-28 14:05:34 +08:00
wushiai1109 29be360954 Update nodes.py (#242)
SELECTED_SEARCH_ENGINE impossible equal to SearchEngine.ARXIV, should be SearchEngine.ARXIV.value, or use the encapsulated get_web_search_tool
2025-05-27 18:58:14 +08:00
Harsha Vardhan Mannem 3ed70e11d5 Fix/server error handling (#212)
* chore: add venv/ to gitignore

* fix: add server error handling and graceful shutdown

* Fix linting issues in server.py
2025-05-22 13:45:07 +08:00
laundry 55ce399969 test: add background node unit test (#198)
* test: add background node unit test

Change-Id: Ia99f5a1687464387dcb01bbee04deaa371c6e490

* test: add background node unit test

Change-Id: I9aabcf02ff04fda40c56f3ea22abe6b8f93bf9b6

* test: fix test error

Change-Id: I3997dc53a2cfaa35501a1fbda5902ee15528124e

* test: fix unit test error

Change-Id: If4c4cd10673e76a30945674c7cda198aeabf28d0

* test: fix unit test error

Change-Id: I3dd7a6179132e5497a30ada443d88de0c47af3d4
2025-05-20 14:25:35 +08:00
DanielWalnut 8bbcdbe4de feat: config max_search_results for search engine (#192)
* feat: implement UI

* feat: config max_search_results for search engine via api

---------

Co-authored-by: Henry Li <henry1943@163.com>
2025-05-18 13:23:52 +08:00
changqingla c6bbc595c3 Fix :This PR can resolve the issue of exceeding the default tool invocation limit by setting the recursion limit through an environment variable.mit (#138)
* set ecursion limit

* set ecursion limit

* fix:check if the recession_limit within a reasonalbe range

* style: format code with black
2025-05-17 20:37:03 -07:00
牧毅 ffe706d0df Allow concurrently run server.py and web in production mode. (#25)
* Allow concurrently run server.py and web in production mode.

* Allow concurrently run server.py and web in production mode.

* Allow concurrently run server.py and web in production mode.
2025-05-17 20:33:00 -07:00
DanielWalnut f7d79b6d83 refactor: upgrade langgraph version (#148) 2025-05-18 11:29:41 +08:00
cndoit18 d69128495b feat: add .venv to dockerignore and optimize Dockerfile with cache mounts for uv (#145)
Change-Id: I27ff2d4f9bcdedbd0135e109ecb6aa6d78bc488b
2025-05-17 21:21:55 +08:00
Willem Jiang 9dc78c3829 fix:added the Portuguese README entry to the README files (#184) 2025-05-16 21:43:01 +08:00
Ernâni de Britto Murtinho 96fb5d653b Added Portuguese pt-br Readme File Version (#127) 2025-05-16 21:10:17 +08:00
Wang Hao e27c43f005 fix: add model_dump (#137)
Co-authored-by: Willem Jiang <143703838+willem-bd@users.noreply.github.com>
2025-05-16 21:05:46 +08:00
hao-cyber c3886e635d docs: add Spanish and Russian translations for README (#183) 2025-05-16 20:56:04 +08:00
Zhengbin Sun c046d9cc34 fix: update responsive design calculations for chat layout (#168) 2025-05-16 11:40:26 +08:00
XingLiu0923 9cff113862 feat(ut): add ut coverage check (#170) 2025-05-15 08:56:13 -07:00
178 changed files with 53176 additions and 2205 deletions
+1
View File
@@ -26,6 +26,7 @@ wheels/
*.egg-info/
.installed.cfg
*.egg
.venv/
# Web
node_modules
+25
View File
@@ -5,12 +5,37 @@ APP_ENV=development
# docker build args
NEXT_PUBLIC_API_URL="http://localhost:8000/api"
AGENT_RECURSION_LIMIT=30
# CORS settings
# Comma-separated list of allowed origins for CORS requests
# Example: ALLOWED_ORIGINS=http://localhost:3000,http://example.com
ALLOWED_ORIGINS=http://localhost:3000
# Enable or disable MCP server configuration, the default is false.
# Please enable this feature before securing your front-end and back-end in a managed environment.
# Otherwise, you system could be compromised.
ENABLE_MCP_SERVER_CONFIGURATION=false
# Search Engine, Supported values: tavily (recommended), duckduckgo, brave_search, arxiv
SEARCH_API=tavily
TAVILY_API_KEY=tvly-xxx
# BRAVE_SEARCH_API_KEY=xxx # Required only if SEARCH_API is brave_search
# JINA_API_KEY=jina_xxx # Optional, default is None
# Optional, RAG provider
# RAG_PROVIDER=vikingdb_knowledge_base
# VIKINGDB_KNOWLEDGE_BASE_API_URL="api-knowledgebase.mlp.cn-beijing.volces.com"
# VIKINGDB_KNOWLEDGE_BASE_API_AK="AKxxx"
# VIKINGDB_KNOWLEDGE_BASE_API_SK=""
# VIKINGDB_KNOWLEDGE_BASE_RETRIEVAL_SIZE=15
# RAG_PROVIDER=ragflow
# RAGFLOW_API_URL="http://localhost:9388"
# RAGFLOW_API_KEY="ragflow-xxx"
# RAGFLOW_RETRIEVAL_SIZE=10
# Optional, volcengine TTS for generating podcast
VOLCENGINE_TTS_APPID=xxx
VOLCENGINE_TTS_ACCESS_TOKEN=xxx
+95
View File
@@ -0,0 +1,95 @@
name: Publish Containers
on:
push:
branches:
- main
release:
types: [published]
workflow_dispatch:
jobs:
backend-container:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
attestations: write
id-token: write
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
id: push
uses: docker/build-push-action@v6
with:
context: .
file: Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Generate artifact attestation
uses: actions/attest-build-provenance@v2
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
frontend-container:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
attestations: write
id-token: write
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}-web
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 #v3.4.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 #v5.7.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
id: push
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 #v6.18.0
with:
context: web
file: web/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Generate artifact attestation
uses: actions/attest-build-provenance@v2
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
+4 -1
View File
@@ -6,6 +6,9 @@ on:
pull_request:
branches: [ '*' ]
permissions:
contents: read
jobs:
lint:
runs-on: ubuntu-latest
@@ -13,7 +16,7 @@ jobs:
- uses: actions/checkout@v3
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
uses: astral-sh/setup-uv@v6.3.1
with:
version: "latest"
+22 -3
View File
@@ -6,6 +6,9 @@ on:
pull_request:
branches: [ '*' ]
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
@@ -13,7 +16,7 @@ jobs:
- uses: actions/checkout@v3
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
uses: astral-sh/setup-uv@v6.3.1
with:
version: "latest"
@@ -23,7 +26,23 @@ jobs:
uv pip install -e ".[dev]"
uv pip install -e ".[test]"
- name: Run test cases
- name: Run test cases with coverage
run: |
source .venv/bin/activate
TAVILY_API_KEY=mock-key make test
TAVILY_API_KEY=mock-key make coverage
- name: Generate HTML Coverage Report
run: |
source .venv/bin/activate
python -m coverage html -d coverage_html
- name: Upload Coverage Report
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage_html/
- name: Display Coverage Summary
run: |
source .venv/bin/activate
python -m coverage report
+5
View File
@@ -6,11 +6,13 @@ dist/
wheels/
*.egg-info
.coverage
.coverage.*
agent_history.gif
static/browser_history/*.gif
# Virtual environments
.venv
venv/
# Environment variables
.env
@@ -21,3 +23,6 @@ conf.yaml
.idea/
.langgraph_api/
# coverage report
coverage.xml
coverage/
+30
View File
@@ -1,6 +1,36 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Debug Tests",
"type": "debugpy",
"request": "launch",
"module": "pytest",
"args": [
"${workspaceFolder}/tests",
"-v",
"-s"
],
"console": "integratedTerminal",
"justMyCode": false,
"env": {
"PYTHONPATH": "${workspaceFolder}"
}
},
{
"name": "Debug Current Test File",
"type": "debugpy",
"request": "launch",
"module": "pytest",
"args": [
"${file}",
"-v",
"-s"
],
"console": "integratedTerminal",
"justMyCode": false
},
{
"name": "Python: 当前文件",
"type": "debugpy",
+7
View File
@@ -0,0 +1,7 @@
{
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true
}
+11
View File
@@ -17,11 +17,14 @@ There are many ways you can contribute to DeerFlow:
1. Fork the repository
2. Clone your fork:
```bash
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
```
3. Set up your development environment:
```bash
# Install dependencies, uv will take care of the python interpreter and venv creation
uv sync
@@ -30,7 +33,9 @@ There are many ways you can contribute to DeerFlow:
uv pip install -e ".[dev]"
uv pip install -e ".[test]"
```
4. Configure pre-commit hooks:
```bash
chmod +x pre-commit
ln -s ../../pre-commit .git/hooks/pre-commit
@@ -39,6 +44,7 @@ There are many ways you can contribute to DeerFlow:
## Development Process
1. Create a new branch:
```bash
git checkout -b feature/amazing-feature
```
@@ -50,6 +56,7 @@ There are many ways you can contribute to DeerFlow:
- Update documentation as needed
3. Run tests and checks:
```bash
make test # Run tests
make lint # Run linting
@@ -58,11 +65,13 @@ There are many ways you can contribute to DeerFlow:
```
4. Commit your changes:
```bash
git commit -m 'Add some amazing feature'
```
5. Push to your fork:
```bash
git push origin feature/amazing-feature
```
@@ -90,6 +99,7 @@ There are many ways you can contribute to DeerFlow:
## Testing
Run the test suite:
```bash
# Run all tests
make test
@@ -122,6 +132,7 @@ make format
## Need Help?
If you need help with anything:
- Check existing issues and discussions
- Join our community channels
- Ask questions in discussions
+9 -2
View File
@@ -1,15 +1,22 @@
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim
FROM ghcr.io/astral-sh/uv:python3.12-bookworm
# Install uv.
COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv
WORKDIR /app
# Pre-cache the application dependencies.
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,source=uv.lock,target=uv.lock \
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
uv sync --locked --no-install-project
# Copy the application into the container.
COPY . /app
# Install the application dependencies.
RUN uv sync --frozen --no-cache
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked
EXPOSE 8000
+2 -1
View File
@@ -8,6 +8,7 @@ format:
lint:
uv run black --check .
uv run ruff check .
serve:
uv run server.py --reload
@@ -19,4 +20,4 @@ langgraph-dev:
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.12 langgraph dev --allow-blocking
coverage:
uv run pytest --cov=src tests/ --cov-report=term-missing
uv run pytest --cov=src tests/ --cov-report=term-missing --cov-report=xml
+30 -8
View File
@@ -2,23 +2,26 @@
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McDcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McCcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
<!-- DeepWiki badge generated by https://deepwiki.ryoppippi.com/ -->
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md)
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) | [Portuguese](./README_pt.md)
> Originated from Open Source, give back to Open Source.
**DeerFlow** (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) is a community-driven Deep Research framework that builds upon the incredible work of the open source community. Our goal is to combine language models with specialized tools for tasks like web search, crawling, and Python code execution, while giving back to the community that made this possible.
Currently, DeerFlow has officially entered the [FaaS Application Center of Volcengine](https://console.volcengine.com/vefaas/region:vefaas+cn-beijing/market). Users can experience it online through the [experience link](https://console.volcengine.com/vefaas/region:vefaas+cn-beijing/market/deerflow/?channel=github&source=deerflow) to intuitively feel its powerful functions and convenient operations. At the same time, to meet the deployment needs of different users, DeerFlow supports one-click deployment based on Volcengine. Click the [deployment link](https://console.volcengine.com/vefaas/region:vefaas+cn-beijing/application/create?templateId=683adf9e372daa0008aaed5c&channel=github&source=deerflow) to quickly complete the deployment process and start an efficient research journey.
Please visit [our official website](https://deerflow.tech/) for more details.
## Demo
### Video
https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
In this demo, we showcase how to use DeerFlow to:
@@ -144,19 +147,18 @@ Explore more details in the [`web`](./web/) directory.
## Supported Search Engines
### Web Search
DeerFlow supports multiple search engines that can be configured in your `.env` file using the `SEARCH_API` variable:
- **Tavily** (default): A specialized search API for AI applications
- Requires `TAVILY_API_KEY` in your `.env` file
- Sign up at: https://app.tavily.com/home
- **DuckDuckGo**: Privacy-focused search engine
- No API key required
- **Brave Search**: Privacy-focused search engine with advanced features
- Requires `BRAVE_SEARCH_API_KEY` in your `.env` file
- Sign up at: https://brave.com/search/api/
@@ -171,6 +173,19 @@ To configure your preferred search engine, set the `SEARCH_API` variable in your
SEARCH_API=tavily
```
### Private Knowledgebase
DeerFlow support private knowledgebase such as ragflow and vikingdb, so that you can use your private documents to answer questions.
- **[RAGFlow](https://ragflow.io/docs/dev/)**open source RAG engine
```
# examples in .env.example
RAG_PROVIDER=ragflow
RAGFLOW_API_URL="http://localhost:9388"
RAGFLOW_API_KEY="ragflow-xxx"
RAGFLOW_RETRIEVAL_SIZE=10
```
## Features
### Core Capabilities
@@ -184,10 +199,14 @@ SEARCH_API=tavily
### Tools and MCP Integrations
- 🔍 **Search and Retrieval**
- Web search via Tavily, Brave Search and more
- Crawling with Jina
- Advanced content extraction
- Support for private knowledgebase
- 📃 **RAG Integration**
- Supports mentioning files from [RAGFlow](https://github.com/infiniflow/ragflow) within the input box. [Start up RAGFlow server](https://ragflow.io/docs/dev/).
- 🔗 **MCP Seamless Integration**
- Expand capabilities for private domain access, knowledge graph, web browsing and more
@@ -196,7 +215,6 @@ SEARCH_API=tavily
### Human Collaboration
- 🧠 **Human-in-the-loop**
- Supports interactive modification of research plans using natural language
- Supports auto-acceptance of research plans
@@ -352,6 +370,7 @@ When you submit a research topic in the Studio UI, you'll be able to see the ent
DeerFlow supports LangSmith tracing to help you debug and monitor your workflows. To enable LangSmith tracing:
1. Make sure your `.env` file has the following configurations (see `.env.example`):
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
@@ -503,6 +522,7 @@ DeerFlow includes a human in the loop mechanism that allows you to review, edit,
- Via API: Set `auto_accepted_plan: true` in your request
4. **API Integration**: When using the API, you can provide feedback through the `feedback` parameter:
```json
{
"messages": [{ "role": "user", "content": "What is quantum computing?" }],
@@ -538,6 +558,8 @@ We would like to extend our sincere appreciation to the following projects for t
- **[LangChain](https://github.com/langchain-ai/langchain)**: Their exceptional framework powers our LLM interactions and chains, enabling seamless integration and functionality.
- **[LangGraph](https://github.com/langchain-ai/langgraph)**: Their innovative approach to multi-agent orchestration has been instrumental in enabling DeerFlow's sophisticated workflows.
- **[Novel](https://github.com/steven-tey/novel)**: Their Notion-style WYSIWYG editor supports our report editing and AI-assisted rewriting.
- **[RAGFlow](https://github.com/infiniflow/ragflow)**: We have achieved support for research on users' private knowledge bases through integration with RAGFlow.
These projects exemplify the transformative power of open-source collaboration, and we are proud to build upon their foundations.
+45 -36
View File
@@ -2,24 +2,27 @@
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McDcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McCcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
<!-- DeepWiki badge generated by https://deepwiki.ryoppippi.com/ -->
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md)
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) | [Portuguese](./README_pt.md)
> Aus Open Source entstanden, an Open Source zurückgeben.
**DeerFlow** (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) ist ein Community-getriebenes Framework für tiefgehende Recherche, das auf der großartigen Arbeit der Open-Source-Community aufbaut. Unser Ziel ist es, Sprachmodelle mit spezialisierten Werkzeugen für Aufgaben wie Websuche, Crawling und Python-Code-Ausführung zu kombinieren und gleichzeitig der Community, die dies möglich gemacht hat, etwas zurückzugeben.
Derzeit ist DeerFlow offiziell in das FaaS-Anwendungszentrum von Volcengine eingezogen. Benutzer können es über den Erfahrungslink online erleben, um seine leistungsstarken Funktionen und bequemen Operationen intuitiv zu spüren. Gleichzeitig unterstützt DeerFlow zur Erfüllung der Bereitstellungsanforderungen verschiedener Benutzer die Ein-Klick-Bereitstellung basierend auf Volcengine. Klicken Sie auf den Bereitstellungslink, um den Bereitstellungsprozess schnell abzuschließen und eine effiziente Forschungsreise zu beginnen.
Besuchen Sie [unsere offizielle Website](https://deerflow.tech/) für weitere Details.
## Demo
### Video
https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
In dieser Demo zeigen wir, wie man DeerFlow nutzt, um:
- Nahtlos mit MCP-Diensten zu integrieren
- Den Prozess der tiefgehenden Recherche durchzuführen und einen umfassenden Bericht mit Bildern zu erstellen
- Podcast-Audio basierend auf dem generierten Bericht zu erstellen
@@ -34,7 +37,6 @@ In dieser Demo zeigen wir, wie man DeerFlow nutzt, um:
---
## 📑 Inhaltsverzeichnis
- [🚀 Schnellstart](#schnellstart)
@@ -48,12 +50,12 @@ In dieser Demo zeigen wir, wie man DeerFlow nutzt, um:
- [💖 Danksagungen](#danksagungen)
- [⭐ Star-Verlauf](#star-verlauf)
## Schnellstart
DeerFlow ist in Python entwickelt und kommt mit einer in Node.js geschriebenen Web-UI. Um einen reibungslosen Einrichtungsprozess zu gewährleisten, empfehlen wir die Verwendung der folgenden Tools:
### Empfohlene Tools
- **[`uv`](https://docs.astral.sh/uv/getting-started/installation/):**
Vereinfacht die Verwaltung von Python-Umgebungen und Abhängigkeiten. `uv` erstellt automatisch eine virtuelle Umgebung im Stammverzeichnis und installiert alle erforderlichen Pakete für Sie—keine manuelle Installation von Python-Umgebungen notwendig.
@@ -64,11 +66,14 @@ DeerFlow ist in Python entwickelt und kommt mit einer in Node.js geschriebenen W
Installieren und verwalten Sie Abhängigkeiten des Node.js-Projekts.
### Umgebungsanforderungen
Stellen Sie sicher, dass Ihr System die folgenden Mindestanforderungen erfüllt:
- **[Python](https://www.python.org/downloads/):** Version `3.12+`
- **[Node.js](https://nodejs.org/en/download/):** Version `22+`
### Installation
```bash
# Repository klonen
git clone https://github.com/bytedance/deer-flow.git
@@ -136,25 +141,24 @@ bootstrap.bat -d
Weitere Details finden Sie im Verzeichnis [`web`](./web/).
## Unterstützte Suchmaschinen
DeerFlow unterstützt mehrere Suchmaschinen, die in Ihrer `.env`-Datei über die Variable `SEARCH_API` konfiguriert werden können:
- **Tavily** (Standard): Eine spezialisierte Such-API für KI-Anwendungen
- Erfordert `TAVILY_API_KEY` in Ihrer `.env`-Datei
- Registrieren Sie sich unter: https://app.tavily.com/home
- Erfordert `TAVILY_API_KEY` in Ihrer `.env`-Datei
- Registrieren Sie sich unter: <https://app.tavily.com/home>
- **DuckDuckGo**: Datenschutzorientierte Suchmaschine
- Kein API-Schlüssel erforderlich
- Kein API-Schlüssel erforderlich
- **Brave Search**: Datenschutzorientierte Suchmaschine mit erweiterten Funktionen
- Erfordert `BRAVE_SEARCH_API_KEY` in Ihrer `.env`-Datei
- Registrieren Sie sich unter: https://brave.com/search/api/
- Erfordert `BRAVE_SEARCH_API_KEY` in Ihrer `.env`-Datei
- Registrieren Sie sich unter: <https://brave.com/search/api/>
- **Arxiv**: Wissenschaftliche Papiersuche für akademische Forschung
- Kein API-Schlüssel erforderlich
- Spezialisiert auf wissenschaftliche und akademische Papiere
- Kein API-Schlüssel erforderlich
- Spezialisiert auf wissenschaftliche und akademische Papiere
Um Ihre bevorzugte Suchmaschine zu konfigurieren, setzen Sie die Variable `SEARCH_API` in Ihrer `.env`-Datei:
@@ -168,40 +172,39 @@ SEARCH_API=tavily
### Kernfähigkeiten
- 🤖 **LLM-Integration**
- Unterstützt die Integration der meisten Modelle über [litellm](https://docs.litellm.ai/docs/providers).
- Unterstützung für Open-Source-Modelle wie Qwen
- OpenAI-kompatible API-Schnittstelle
- Mehrstufiges LLM-System für unterschiedliche Aufgabenkomplexitäten
- Unterstützt die Integration der meisten Modelle über [litellm](https://docs.litellm.ai/docs/providers).
- Unterstützung für Open-Source-Modelle wie Qwen
- OpenAI-kompatible API-Schnittstelle
- Mehrstufiges LLM-System für unterschiedliche Aufgabenkomplexitäten
### Tools und MCP-Integrationen
- 🔍 **Suche und Abruf**
- Websuche über Tavily, Brave Search und mehr
- Crawling mit Jina
- Fortgeschrittene Inhaltsextraktion
- Websuche über Tavily, Brave Search und mehr
- Crawling mit Jina
- Fortgeschrittene Inhaltsextraktion
- 🔗 **MCP Nahtlose Integration**
- Erweiterte Fähigkeiten für privaten Domänenzugriff, Wissensgraphen, Webbrowsing und mehr
- Erleichtert die Integration verschiedener Forschungswerkzeuge und -methoden
- Erweiterte Fähigkeiten für privaten Domänenzugriff, Wissensgraphen, Webbrowsing und mehr
- Erleichtert die Integration verschiedener Forschungswerkzeuge und -methoden
### Menschliche Zusammenarbeit
- 🧠 **Mensch-in-der-Schleife**
- Unterstützt interaktive Modifikation von Forschungsplänen mit natürlicher Sprache
- Unterstützt automatische Akzeptanz von Forschungsplänen
- Unterstützt interaktive Modifikation von Forschungsplänen mit natürlicher Sprache
- Unterstützt automatische Akzeptanz von Forschungsplänen
- 📝 **Bericht-Nachbearbeitung**
- Unterstützt Notion-ähnliche Blockbearbeitung
- Ermöglicht KI-Verfeinerungen, einschließlich KI-unterstützter Polierung, Satzkürzung und -erweiterung
- Angetrieben von [tiptap](https://tiptap.dev/)
- Unterstützt Notion-ähnliche Blockbearbeitung
- Ermöglicht KI-Verfeinerungen, einschließlich KI-unterstützter Polierung, Satzkürzung und -erweiterung
- Angetrieben von [tiptap](https://tiptap.dev/)
### Inhaltserstellung
- 🎙️ **Podcast- und Präsentationserstellung**
- KI-gestützte Podcast-Skripterstellung und Audiosynthese
- Automatisierte Erstellung einfacher PowerPoint-Präsentationen
- Anpassbare Vorlagen für maßgeschneiderte Inhalte
- KI-gestützte Podcast-Skripterstellung und Audiosynthese
- Automatisierte Erstellung einfacher PowerPoint-Präsentationen
- Anpassbare Vorlagen für maßgeschneiderte Inhalte
## Architektur
@@ -253,7 +256,6 @@ curl --location 'http://localhost:8000/api/tts' \
--output speech.mp3
```
## Entwicklung
### Testen
@@ -311,9 +313,10 @@ langgraph dev
```
Nach dem Start des LangGraph-Servers sehen Sie mehrere URLs im Terminal:
- API: http://127.0.0.1:2024
- Studio UI: https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024
- API-Dokumentation: http://127.0.0.1:2024/docs
- API: <http://127.0.0.1:2024>
- Studio UI: <https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024>
- API-Dokumentation: <http://127.0.0.1:2024/docs>
Öffnen Sie den Studio UI-Link in Ihrem Browser, um auf die Debugging-Schnittstelle zuzugreifen.
@@ -328,6 +331,7 @@ In der Studio UI können Sie:
5. Feedback während der Planungsphase geben, um Forschungspläne zu verfeinern
Wenn Sie ein Forschungsthema in der Studio UI einreichen, können Sie die gesamte Workflow-Ausführung sehen, einschließlich:
- Die Planungsphase, in der der Forschungsplan erstellt wird
- Die Feedback-Schleife, in der Sie den Plan ändern können
- Die Forschungs- und Schreibphasen für jeden Abschnitt
@@ -338,6 +342,7 @@ Wenn Sie ein Forschungsthema in der Studio UI einreichen, können Sie die gesamt
DeerFlow unterstützt LangSmith-Tracing, um Ihnen beim Debuggen und Überwachen Ihrer Workflows zu helfen. Um LangSmith-Tracing zu aktivieren:
1. Stellen Sie sicher, dass Ihre `.env`-Datei die folgenden Konfigurationen enthält (siehe `.env.example`):
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
@@ -346,6 +351,7 @@ DeerFlow unterstützt LangSmith-Tracing, um Ihnen beim Debuggen und Überwachen
```
2. Starten Sie das Tracing mit LangSmith lokal, indem Sie folgenden Befehl ausführen:
```bash
langgraph dev
```
@@ -419,6 +425,7 @@ uv run main.py --help
Die Anwendung unterstützt jetzt einen interaktiven Modus mit eingebauten Fragen in Englisch und Chinesisch:
1. Starten Sie den interaktiven Modus:
```bash
uv run main.py --interactive
```
@@ -444,6 +451,7 @@ DeerFlow enthält einen Mensch-in-der-Schleife-Mechanismus, der es Ihnen ermögl
- Über API: Setzen Sie `auto_accepted_plan: true` in Ihrer Anfrage
4. **API-Integration**: Bei Verwendung der API können Sie Feedback über den Parameter `feedback` geben:
```json
{
"messages": [{"role": "user", "content": "Was ist Quantencomputing?"}],
@@ -483,6 +491,7 @@ Wir möchten unsere aufrichtige Wertschätzung den folgenden Projekten für ihre
Diese Projekte veranschaulichen die transformative Kraft der Open-Source-Zusammenarbeit, und wir sind stolz darauf, auf ihren Grundlagen aufzubauen.
### Hauptmitwirkende
Ein herzliches Dankeschön geht an die Hauptautoren von `DeerFlow`, deren Vision, Leidenschaft und Engagement dieses Projekt zum Leben erweckt haben:
- **[Daniel Walnut](https://github.com/hetaoBackend/)**
@@ -492,4 +501,4 @@ Ihr unerschütterliches Engagement und Fachwissen waren die treibende Kraft hint
## Star-Verlauf
[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
+559
View File
@@ -0,0 +1,559 @@
# 🦌 DeerFlow
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McCcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
<!-- DeepWiki badge generated by https://deepwiki.ryoppippi.com/ -->
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) | [Portuguese](./README_pt.md)
> Originado del código abierto, retribuido al código abierto.
**DeerFlow** (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) es un marco de Investigación Profunda impulsado por la comunidad que se basa en el increíble trabajo de la comunidad de código abierto. Nuestro objetivo es combinar modelos de lenguaje con herramientas especializadas para tareas como búsqueda web, rastreo y ejecución de código Python, mientras devolvemos a la comunidad que hizo esto posible.
Actualmente, DeerFlow ha ingresado oficialmente al Centro de Aplicaciones FaaS de Volcengine. Los usuarios pueden experimentarlo en línea a través del enlace de experiencia para sentir intuitivamente sus potentes funciones y operaciones convenientes. Al mismo tiempo, para satisfacer las necesidades de implementación de diferentes usuarios, DeerFlow admite la implementación con un clic basada en Volcengine. Haga clic en el enlace de implementación para completar rápidamente el proceso de implementación y comenzar un viaje de investigación eficiente.
Por favor, visita [nuestra página web oficial](https://deerflow.tech/) para más detalles.
## Demostración
### Video
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
En esta demostración, mostramos cómo usar DeerFlow para:
- Integrar perfectamente con servicios MCP
- Realizar el proceso de Investigación Profunda y producir un informe completo con imágenes
- Crear audio de podcast basado en el informe generado
### Repeticiones
- [¿Qué altura tiene la Torre Eiffel comparada con el edificio más alto?](https://deerflow.tech/chat?replay=eiffel-tower-vs-tallest-building)
- [¿Cuáles son los repositorios más populares en GitHub?](https://deerflow.tech/chat?replay=github-top-trending-repo)
- [Escribir un artículo sobre los platos tradicionales de Nanjing](https://deerflow.tech/chat?replay=nanjing-traditional-dishes)
- [¿Cómo decorar un apartamento de alquiler?](https://deerflow.tech/chat?replay=rental-apartment-decoration)
- [Visita nuestra página web oficial para explorar más repeticiones.](https://deerflow.tech/#case-studies)
---
## 📑 Tabla de Contenidos
- [🚀 Inicio Rápido](#inicio-rápido)
- [🌟 Características](#características)
- [🏗️ Arquitectura](#arquitectura)
- [🛠️ Desarrollo](#desarrollo)
- [🐳 Docker](#docker)
- [🗣️ Integración de Texto a Voz](#integración-de-texto-a-voz)
- [📚 Ejemplos](#ejemplos)
- [❓ Preguntas Frecuentes](#preguntas-frecuentes)
- [📜 Licencia](#licencia)
- [💖 Agradecimientos](#agradecimientos)
- [⭐ Historial de Estrellas](#historial-de-estrellas)
## Inicio Rápido
DeerFlow está desarrollado en Python y viene con una interfaz web escrita en Node.js. Para garantizar un proceso de configuración sin problemas, recomendamos utilizar las siguientes herramientas:
### Herramientas Recomendadas
- **[`uv`](https://docs.astral.sh/uv/getting-started/installation/):**
Simplifica la gestión del entorno Python y las dependencias. `uv` crea automáticamente un entorno virtual en el directorio raíz e instala todos los paquetes necesarios por ti—sin necesidad de instalar entornos Python manualmente.
- **[`nvm`](https://github.com/nvm-sh/nvm):**
Gestiona múltiples versiones del entorno de ejecución Node.js sin esfuerzo.
- **[`pnpm`](https://pnpm.io/installation):**
Instala y gestiona dependencias del proyecto Node.js.
### Requisitos del Entorno
Asegúrate de que tu sistema cumple con los siguientes requisitos mínimos:
- **[Python](https://www.python.org/downloads/):** Versión `3.12+`
- **[Node.js](https://nodejs.org/en/download/):** Versión `22+`
### Instalación
```bash
# Clonar el repositorio
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
# Instalar dependencias, uv se encargará del intérprete de python, la creación del entorno virtual y la instalación de los paquetes necesarios
uv sync
# Configurar .env con tus claves API
# Tavily: https://app.tavily.com/home
# Brave_SEARCH: https://brave.com/search/api/
# volcengine TTS: Añade tus credenciales TTS si las tienes
cp .env.example .env
# Ver las secciones 'Motores de Búsqueda Compatibles' e 'Integración de Texto a Voz' a continuación para todas las opciones disponibles
# Configurar conf.yaml para tu modelo LLM y claves API
# Por favor, consulta 'docs/configuration_guide.md' para más detalles
cp conf.yaml.example conf.yaml
# Instalar marp para la generación de presentaciones
# https://github.com/marp-team/marp-cli?tab=readme-ov-file#use-package-manager
brew install marp-cli
```
Opcionalmente, instala las dependencias de la interfaz web vía [pnpm](https://pnpm.io/installation):
```bash
cd deer-flow/web
pnpm install
```
### Configuraciones
Por favor, consulta la [Guía de Configuración](docs/configuration_guide.md) para más detalles.
> [!NOTA]
> Antes de iniciar el proyecto, lee la guía cuidadosamente y actualiza las configuraciones para que coincidan con tus ajustes y requisitos específicos.
### Interfaz de Consola
La forma más rápida de ejecutar el proyecto es utilizar la interfaz de consola.
```bash
# Ejecutar el proyecto en un shell tipo bash
uv run main.py
```
### Interfaz Web
Este proyecto también incluye una Interfaz Web, que ofrece una experiencia interactiva más dinámica y atractiva.
> [!NOTA]
> Necesitas instalar primero las dependencias de la interfaz web.
```bash
# Ejecutar tanto el servidor backend como el frontend en modo desarrollo
# En macOS/Linux
./bootstrap.sh -d
# En Windows
bootstrap.bat -d
```
Abre tu navegador y visita [`http://localhost:3000`](http://localhost:3000) para explorar la interfaz web.
Explora más detalles en el directorio [`web`](./web/).
## Motores de Búsqueda Compatibles
DeerFlow soporta múltiples motores de búsqueda que pueden configurarse en tu archivo `.env` usando la variable `SEARCH_API`:
- **Tavily** (predeterminado): Una API de búsqueda especializada para aplicaciones de IA
- Requiere `TAVILY_API_KEY` en tu archivo `.env`
- Regístrate en: <https://app.tavily.com/home>
- **DuckDuckGo**: Motor de búsqueda centrado en la privacidad
- No requiere clave API
- **Brave Search**: Motor de búsqueda centrado en la privacidad con características avanzadas
- Requiere `BRAVE_SEARCH_API_KEY` en tu archivo `.env`
- Regístrate en: <https://brave.com/search/api/>
- **Arxiv**: Búsqueda de artículos científicos para investigación académica
- No requiere clave API
- Especializado en artículos científicos y académicos
Para configurar tu motor de búsqueda preferido, establece la variable `SEARCH_API` en tu archivo `.env`:
```bash
# Elige uno: tavily, duckduckgo, brave_search, arxiv
SEARCH_API=tavily
```
## Características
### Capacidades Principales
- 🤖 **Integración de LLM**
- Soporta la integración de la mayoría de los modelos a través de [litellm](https://docs.litellm.ai/docs/providers).
- Soporte para modelos de código abierto como Qwen
- Interfaz API compatible con OpenAI
- Sistema LLM de múltiples niveles para diferentes complejidades de tareas
### Herramientas e Integraciones MCP
- 🔍 **Búsqueda y Recuperación**
- Búsqueda web a través de Tavily, Brave Search y más
- Rastreo con Jina
- Extracción avanzada de contenido
- 🔗 **Integración Perfecta con MCP**
- Amplía capacidades para acceso a dominio privado, gráfico de conocimiento, navegación web y más
- Facilita la integración de diversas herramientas y metodologías de investigación
### Colaboración Humana
- 🧠 **Humano en el Bucle**
- Soporta modificación interactiva de planes de investigación usando lenguaje natural
- Soporta aceptación automática de planes de investigación
- 📝 **Post-Edición de Informes**
- Soporta edición de bloques tipo Notion
- Permite refinamientos por IA, incluyendo pulido asistido por IA, acortamiento y expansión de oraciones
- Impulsado por [tiptap](https://tiptap.dev/)
### Creación de Contenido
- 🎙️ **Generación de Podcasts y Presentaciones**
- Generación de guiones de podcast y síntesis de audio impulsadas por IA
- Creación automatizada de presentaciones PowerPoint simples
- Plantillas personalizables para contenido a medida
## Arquitectura
DeerFlow implementa una arquitectura modular de sistema multi-agente diseñada para investigación automatizada y análisis de código. El sistema está construido sobre LangGraph, permitiendo un flujo de trabajo flexible basado en estados donde los componentes se comunican a través de un sistema de paso de mensajes bien definido.
![Diagrama de Arquitectura](./assets/architecture.png)
> Vélo en vivo en [deerflow.tech](https://deerflow.tech/#multi-agent-architecture)
El sistema emplea un flujo de trabajo racionalizado con los siguientes componentes:
1. **Coordinador**: El punto de entrada que gestiona el ciclo de vida del flujo de trabajo
- Inicia el proceso de investigación basado en la entrada del usuario
- Delega tareas al planificador cuando corresponde
- Actúa como la interfaz principal entre el usuario y el sistema
2. **Planificador**: Componente estratégico para descomposición y planificación de tareas
- Analiza objetivos de investigación y crea planes de ejecución estructurados
- Determina si hay suficiente contexto disponible o si se necesita más investigación
- Gestiona el flujo de investigación y decide cuándo generar el informe final
3. **Equipo de Investigación**: Una colección de agentes especializados que ejecutan el plan:
- **Investigador**: Realiza búsquedas web y recopilación de información utilizando herramientas como motores de búsqueda web, rastreo e incluso servicios MCP.
- **Programador**: Maneja análisis de código, ejecución y tareas técnicas utilizando la herramienta Python REPL.
Cada agente tiene acceso a herramientas específicas optimizadas para su rol y opera dentro del marco LangGraph
4. **Reportero**: Procesador de etapa final para los resultados de la investigación
- Agrega hallazgos del equipo de investigación
- Procesa y estructura la información recopilada
- Genera informes de investigación completos
## Integración de Texto a Voz
DeerFlow ahora incluye una función de Texto a Voz (TTS) que te permite convertir informes de investigación a voz. Esta función utiliza la API TTS de volcengine para generar audio de alta calidad a partir de texto. Características como velocidad, volumen y tono también son personalizables.
### Usando la API TTS
Puedes acceder a la funcionalidad TTS a través del punto final `/api/tts`:
```bash
# Ejemplo de llamada API usando curl
curl --location 'http://localhost:8000/api/tts' \
--header 'Content-Type: application/json' \
--data '{
"text": "Esto es una prueba de la funcionalidad de texto a voz.",
"speed_ratio": 1.0,
"volume_ratio": 1.0,
"pitch_ratio": 1.0
}' \
--output speech.mp3
```
## Desarrollo
### Pruebas
Ejecuta el conjunto de pruebas:
```bash
# Ejecutar todas las pruebas
make test
# Ejecutar archivo de prueba específico
pytest tests/integration/test_workflow.py
# Ejecutar con cobertura
make coverage
```
### Calidad del Código
```bash
# Ejecutar linting
make lint
# Formatear código
make format
```
### Depuración con LangGraph Studio
DeerFlow utiliza LangGraph para su arquitectura de flujo de trabajo. Puedes usar LangGraph Studio para depurar y visualizar el flujo de trabajo en tiempo real.
#### Ejecutando LangGraph Studio Localmente
DeerFlow incluye un archivo de configuración `langgraph.json` que define la estructura del grafo y las dependencias para LangGraph Studio. Este archivo apunta a los grafos de flujo de trabajo definidos en el proyecto y carga automáticamente variables de entorno desde el archivo `.env`.
##### Mac
```bash
# Instala el gestor de paquetes uv si no lo tienes
curl -LsSf https://astral.sh/uv/install.sh | sh
# Instala dependencias e inicia el servidor LangGraph
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.12 langgraph dev --allow-blocking
```
##### Windows / Linux
```bash
# Instalar dependencias
pip install -e .
pip install -U "langgraph-cli[inmem]"
# Iniciar el servidor LangGraph
langgraph dev
```
Después de iniciar el servidor LangGraph, verás varias URLs en la terminal:
- API: <http://127.0.0.1:2024>
- UI de Studio: <https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024>
- Docs de API: <http://127.0.0.1:2024/docs>
Abre el enlace de UI de Studio en tu navegador para acceder a la interfaz de depuración.
#### Usando LangGraph Studio
En la UI de Studio, puedes:
1. Visualizar el grafo de flujo de trabajo y ver cómo se conectan los componentes
2. Rastrear la ejecución en tiempo real para ver cómo fluyen los datos a través del sistema
3. Inspeccionar el estado en cada paso del flujo de trabajo
4. Depurar problemas examinando entradas y salidas de cada componente
5. Proporcionar retroalimentación durante la fase de planificación para refinar planes de investigación
Cuando envías un tema de investigación en la UI de Studio, podrás ver toda la ejecución del flujo de trabajo, incluyendo:
- La fase de planificación donde se crea el plan de investigación
- El bucle de retroalimentación donde puedes modificar el plan
- Las fases de investigación y escritura para cada sección
- La generación del informe final
### Habilitando el Rastreo de LangSmith
DeerFlow soporta el rastreo de LangSmith para ayudarte a depurar y monitorear tus flujos de trabajo. Para habilitar el rastreo de LangSmith:
1. Asegúrate de que tu archivo `.env` tenga las siguientes configuraciones (ver `.env.example`):
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
LANGSMITH_API_KEY="xxx"
LANGSMITH_PROJECT="xxx"
```
2. Inicia el rastreo y visualiza el grafo localmente con LangSmith ejecutando:
```bash
langgraph dev
```
Esto habilitará la visualización de rastros en LangGraph Studio y enviará tus rastros a LangSmith para monitoreo y análisis.
## Docker
También puedes ejecutar este proyecto con Docker.
Primero, necesitas leer la [configuración](docs/configuration_guide.md) a continuación. Asegúrate de que los archivos `.env` y `.conf.yaml` estén listos.
Segundo, para construir una imagen Docker de tu propio servidor web:
```bash
docker build -t deer-flow-api .
```
Finalmente, inicia un contenedor Docker que ejecute el servidor web:
```bash
# Reemplaza deer-flow-api-app con tu nombre de contenedor preferido
docker run -d -t -p 8000:8000 --env-file .env --name deer-flow-api-app deer-flow-api
# detener el servidor
docker stop deer-flow-api-app
```
### Docker Compose (incluye tanto backend como frontend)
DeerFlow proporciona una configuración docker-compose para ejecutar fácilmente tanto el backend como el frontend juntos:
```bash
# construir imagen docker
docker compose build
# iniciar el servidor
docker compose up
```
## Ejemplos
Los siguientes ejemplos demuestran las capacidades de DeerFlow:
### Informes de Investigación
1. **Informe sobre OpenAI Sora** - Análisis de la herramienta IA Sora de OpenAI
- Discute características, acceso, ingeniería de prompts, limitaciones y consideraciones éticas
- [Ver informe completo](examples/openai_sora_report.md)
2. **Informe sobre el Protocolo Agent to Agent de Google** - Visión general del protocolo Agent to Agent (A2A) de Google
- Discute su papel en la comunicación de agentes IA y su relación con el Model Context Protocol (MCP) de Anthropic
- [Ver informe completo](examples/what_is_agent_to_agent_protocol.md)
3. **¿Qué es MCP?** - Un análisis completo del término "MCP" en múltiples contextos
- Explora Model Context Protocol en IA, Fosfato Monocálcico en química y Placa de Microcanales en electrónica
- [Ver informe completo](examples/what_is_mcp.md)
4. **Fluctuaciones del Precio de Bitcoin** - Análisis de los movimientos recientes del precio de Bitcoin
- Examina tendencias del mercado, influencias regulatorias e indicadores técnicos
- Proporciona recomendaciones basadas en datos históricos
- [Ver informe completo](examples/bitcoin_price_fluctuation.md)
5. **¿Qué es LLM?** - Una exploración en profundidad de los Modelos de Lenguaje Grandes
- Discute arquitectura, entrenamiento, aplicaciones y consideraciones éticas
- [Ver informe completo](examples/what_is_llm.md)
6. **¿Cómo usar Claude para Investigación Profunda?** - Mejores prácticas y flujos de trabajo para usar Claude en investigación profunda
- Cubre ingeniería de prompts, análisis de datos e integración con otras herramientas
- [Ver informe completo](examples/how_to_use_claude_deep_research.md)
7. **Adopción de IA en Salud: Factores de Influencia** - Análisis de factores que impulsan la adopción de IA en salud
- Discute tecnologías IA, calidad de datos, consideraciones éticas, evaluaciones económicas, preparación organizativa e infraestructura digital
- [Ver informe completo](examples/AI_adoption_in_healthcare.md)
8. **Impacto de la Computación Cuántica en la Criptografía** - Análisis del impacto de la computación cuántica en la criptografía
- Discute vulnerabilidades de la criptografía clásica, criptografía post-cuántica y soluciones criptográficas resistentes a la cuántica
- [Ver informe completo](examples/Quantum_Computing_Impact_on_Cryptography.md)
9. **Aspectos Destacados del Rendimiento de Cristiano Ronaldo** - Análisis de los aspectos destacados del rendimiento de Cristiano Ronaldo
- Discute sus logros profesionales, goles internacionales y rendimiento en varios partidos
- [Ver informe completo](examples/Cristiano_Ronaldo's_Performance_Highlights.md)
Para ejecutar estos ejemplos o crear tus propios informes de investigación, puedes usar los siguientes comandos:
```bash
# Ejecutar con una consulta específica
uv run main.py "¿Qué factores están influyendo en la adopción de IA en salud?"
# Ejecutar con parámetros de planificación personalizados
uv run main.py --max_plan_iterations 3 "¿Cómo impacta la computación cuántica en la criptografía?"
# Ejecutar en modo interactivo con preguntas integradas
uv run main.py --interactive
# O ejecutar con prompt interactivo básico
uv run main.py
# Ver todas las opciones disponibles
uv run main.py --help
```
### Modo Interactivo
La aplicación ahora soporta un modo interactivo con preguntas integradas tanto en inglés como en chino:
1. Lanza el modo interactivo:
```bash
uv run main.py --interactive
```
2. Selecciona tu idioma preferido (English o 中文)
3. Elige de una lista de preguntas integradas o selecciona la opción para hacer tu propia pregunta
4. El sistema procesará tu pregunta y generará un informe de investigación completo
### Humano en el Bucle
DeerFlow incluye un mecanismo de humano en el bucle que te permite revisar, editar y aprobar planes de investigación antes de que sean ejecutados:
1. **Revisión del Plan**: Cuando el humano en el bucle está habilitado, el sistema presentará el plan de investigación generado para tu revisión antes de la ejecución
2. **Proporcionando Retroalimentación**: Puedes:
- Aceptar el plan respondiendo con `[ACCEPTED]`
- Editar el plan proporcionando retroalimentación (p.ej., `[EDIT PLAN] Añadir más pasos sobre implementación técnica`)
- El sistema incorporará tu retroalimentación y generará un plan revisado
3. **Auto-aceptación**: Puedes habilitar la auto-aceptación para omitir el proceso de revisión:
- Vía API: Establece `auto_accepted_plan: true` en tu solicitud
4. **Integración API**: Cuando uses la API, puedes proporcionar retroalimentación a través del parámetro `feedback`:
```json
{
"messages": [{ "role": "user", "content": "¿Qué es la computación cuántica?" }],
"thread_id": "my_thread_id",
"auto_accepted_plan": false,
"feedback": "[EDIT PLAN] Incluir más sobre algoritmos cuánticos"
}
```
### Argumentos de Línea de Comandos
La aplicación soporta varios argumentos de línea de comandos para personalizar su comportamiento:
- **query**: La consulta de investigación a procesar (puede ser múltiples palabras)
- **--interactive**: Ejecutar en modo interactivo con preguntas integradas
- **--max_plan_iterations**: Número máximo de ciclos de planificación (predeterminado: 1)
- **--max_step_num**: Número máximo de pasos en un plan de investigación (predeterminado: 3)
- **--debug**: Habilitar registro detallado de depuración
## Preguntas Frecuentes
Por favor, consulta [FAQ.md](docs/FAQ.md) para más detalles.
## Licencia
Este proyecto es de código abierto y está disponible bajo la [Licencia MIT](./LICENSE).
## Agradecimientos
DeerFlow está construido sobre el increíble trabajo de la comunidad de código abierto. Estamos profundamente agradecidos a todos los proyectos y contribuyentes cuyos esfuerzos han hecho posible DeerFlow. Verdaderamente, nos apoyamos en hombros de gigantes.
Nos gustaría extender nuestro sincero agradecimiento a los siguientes proyectos por sus invaluables contribuciones:
- **[LangChain](https://github.com/langchain-ai/langchain)**: Su excepcional marco impulsa nuestras interacciones y cadenas LLM, permitiendo integración y funcionalidad sin problemas.
- **[LangGraph](https://github.com/langchain-ai/langgraph)**: Su enfoque innovador para la orquestación multi-agente ha sido instrumental en permitir los sofisticados flujos de trabajo de DeerFlow.
Estos proyectos ejemplifican el poder transformador de la colaboración de código abierto, y estamos orgullosos de construir sobre sus cimientos.
### Contribuyentes Clave
Un sentido agradecimiento va para los autores principales de `DeerFlow`, cuya visión, pasión y dedicación han dado vida a este proyecto:
- **[Daniel Walnut](https://github.com/hetaoBackend/)**
- **[Henry Li](https://github.com/magiccube/)**
Su compromiso inquebrantable y experiencia han sido la fuerza impulsora detrás del éxito de DeerFlow. Nos sentimos honrados de tenerlos al timón de este viaje.
## Historial de Estrellas
[![Gráfico de Historial de Estrellas](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
+55 -32
View File
@@ -3,23 +3,25 @@
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md)
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) | [Portuguese](./README_pt.md)
> オープンソースから生まれ、オープンソースに還元する。
**DeerFlow****D**eep **E**xploration and **E**fficient **R**esearch **Flow**)は、オープンソースコミュニティの素晴らしい成果の上に構築されたコミュニティ主導の深層研究フレームワークです。私たちの目標は、言語モデルとウェブ検索、クローリング、Python コード実行などの専門ツールを組み合わせながら、これを可能にしたコミュニティに貢献することです。
現在、DeerFlow は火山引擎の FaaS アプリケーションセンターに正式に入居しています。ユーザーは体験リンクを通じてオンラインで体験し、その強力な機能と便利な操作を直感的に感じることができます。同時に、さまざまなユーザーの展開ニーズを満たすため、DeerFlow は火山引擎に基づくワンクリック展開をサポートしています。展開リンクをクリックして展開プロセスを迅速に完了し、効率的な研究の旅を始めましょう。
詳細については[DeerFlow の公式ウェブサイト](https://deerflow.tech/)をご覧ください。
## デモ
### ビデオ
https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
このデモでは、DeerFlow の使用方法を紹介しています:
このデモでは、DeerFlowの使用方法を紹介しています:
- MCP サービスとのシームレスな統合
- MCPサービスとのシームレスな統合
- 深層研究プロセスの実施と画像を含む包括的なレポートの作成
- 生成されたレポートに基づくポッドキャストオーディオの作成
@@ -143,21 +145,18 @@ bootstrap.bat -d
DeerFlow は複数の検索エンジンをサポートしており、`.env`ファイルの`SEARCH_API`変数で設定できます:
- **Tavily**(デフォルト):AI アプリケーション向けの専門検索 API
- `.env`ファイルに`TAVILY_API_KEY`が必要
- 登録先:https://app.tavily.com/home
- 登録先:<https://app.tavily.com/home>
- **DuckDuckGo**:プライバシー重視の検索エンジン
- API キー不要
- APIキー不要
- **Brave Search**:高度な機能を備えたプライバシー重視の検索エンジン
- `.env`ファイルに`BRAVE_SEARCH_API_KEY`が必要
- 登録先:https://brave.com/search/api/
- 登録先:<https://brave.com/search/api/>
- **Arxiv**:学術研究用の科学論文検索
- API キー不要
- APIキー不要
- 科学・学術論文専用
お好みの検索エンジンを設定するには、`.env`ファイルで`SEARCH_API`変数を設定します:
@@ -171,41 +170,39 @@ SEARCH_API=tavily
### コア機能
- 🤖 **LLM 統合**
- 🤖 **LLM統合**
- [litellm](https://docs.litellm.ai/docs/providers)を通じてほとんどのモデルの統合をサポート
- Qwen などのオープンソースモデルをサポート
- OpenAI 互換の API インターフェース
- 異なるタスクの複雑さに対応するマルチティア LLM システム
- Qwenなどのオープンソースモデルをサポート
- OpenAI互換のAPIインターフェース
- 異なるタスクの複雑さに対応するマルチティアLLMシステム
### ツールと MCP 統合
- 🔍 **検索と取得**
- Tavily、Brave Search などを通じた Web 検索
- Jina を使用したクローリング
- Tavily、Brave Searchなどを通じたWeb検索
- Jinaを使用したクローリング
- 高度なコンテンツ抽出
- 🔗 **MCP シームレス統合**
- プライベートドメインアクセス、ナレッジグラフ、Web ブラウジングなどの機能を拡張
- 🔗 **MCPシームレス統合**
- プライベートドメインアクセス、ナレッジグラフ、Webブラウジングなどの機能を拡張
- 多様な研究ツールと方法論の統合を促進
### 人間との協力
- 🧠 **人間参加型ループ**
- 自然言語を使用した研究計画の対話的修正をサポート
- 研究計画の自動承認をサポート
- 📝 **レポート後編集**
- Notion ライクなブロック編集をサポート
- AI 支援による洗練、文の短縮、拡張などの AI 改良を可能に
- Notionライクなブロック編集をサポート
- AI支援による洗練、文の短縮、拡張などのAI改良を可能に
- [tiptap](https://tiptap.dev/)を活用
### コンテンツ作成
- 🎙️ **ポッドキャストとプレゼンテーション生成**
- AI 駆動のポッドキャストスクリプト生成と音声合成
- シンプルな PowerPoint プレゼンテーションの自動作成
- AI駆動のポッドキャストスクリプト生成と音声合成
- シンプルなPowerPointプレゼンテーションの自動作成
- カスタマイズ可能なテンプレートで個別のコンテンツに対応
## アーキテクチャ
@@ -241,6 +238,27 @@ DeerFlow は、自動研究とコード分析のためのモジュラーなマ
- 収集した情報を処理および構造化
- 包括的な研究レポートを生成
## テキスト読み上げ統合
DeerFlowには現在、研究レポートを音声に変換できるテキスト読み上げ(TTS)機能が含まれています。この機能は火山引擎TTS APIを使用して高品質なテキストオーディオを生成します。速度、音量、ピッチなどの特性もカスタマイズ可能です。
### TTS APIの使用
`/api/tts`エンドポイントからTTS機能にアクセスできます:
```bash
# curlを使用したAPI呼び出し例
curl --location 'http://localhost:8000/api/tts' \
--header 'Content-Type: application/json' \
--data '{
"text": "これはテキスト読み上げ機能のテストです。",
"speed_ratio": 1.0,
"volume_ratio": 1.0,
"pitch_ratio": 1.0
}' \
--output speech.mp3
```
## 開発
### テスト
@@ -297,11 +315,15 @@ pip install -U "langgraph-cli[inmem]"
langgraph dev
```
LangGraph サーバーを開始すると、端末にいくつかの URL が表示されます:
LangGraphサーバーを開始すると、端末にいくつかのURLが表示されます:
- API: http://127.0.0.1:2024
- Studio UI: https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024
- API ドキュメント: http://127.0.0.1:2024/docs
- API: <http://127.0.0.1:2024>
- Studio UI: <https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024>
- APIドキュメント: <http://127.0.0.1:2024/docs>
- API: <http://127.0.0.1:2024>
- Studio UI: <https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024>
- APIドキュメント: <http://127.0.0.1:2024/docs>
ブラウザで Studio UI リンクを開いてデバッグインターフェースにアクセスします。
@@ -315,7 +337,7 @@ Studio UI では、次のことができます:
4. 各コンポーネントの入力と出力を検査して問題をデバッグ
5. 計画段階でフィードバックを提供して研究計画を洗練
Studio UI で研究トピックを送信すると、次を含む全ワークフロー実行プロセスを見ることができます:
Studio UIで研究トピックを送信すると、次を含む全ワークフロー実行プロセスを見ることができます:
- 研究計画を作成する計画段階
- 計画を修正できるフィードバックループ
@@ -327,6 +349,7 @@ Studio UI で研究トピックを送信すると、次を含む全ワークフ
DeerFlow は LangSmith トレース機能をサポートしており、ワークフローのデバッグとモニタリングに役立ちます。LangSmith トレースを有効にするには:
1. `.env` ファイルに次の設定があることを確認してください(`.env.example` を参照):
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
@@ -335,6 +358,7 @@ DeerFlow は LangSmith トレース機能をサポートしており、ワーク
```
2. 次のコマンドを実行して LangSmith トレースを開始します:
```bash
langgraph dev
```
@@ -496,9 +520,8 @@ DeerFlow には人間参加型ループメカニズムが含まれており、
3. **自動承認**:レビュープロセスをスキップするために自動承認を有効にできます:
- API 経由:リクエストで`auto_accepted_plan: true`を設定
4. **API統合**APIを使用する場合、`feedback`パラメータでフィードバックを提供できます:
4. **API 統合**API を使用する場合、`feedback`パラメータでフィードバックを提供できます:
```json
{
"messages": [
+545
View File
@@ -0,0 +1,545 @@
# 🦌 DeerFlow
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McDcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
<!-- DeepWiki badge generated by https://deepwiki.ryoppippi.com/ -->
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) | [Portuguese](./README_pt.md)
> Originado do Open Source, de volta ao Open Source
**DeerFlow** (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) é um framework de Pesquisa Profunda orientado-a-comunidade que baseia-se em um íncrivel trabalho da comunidade open source. Nosso objetivo é combinar modelos de linguagem com ferramentas especializadas para tarefas como busca na web, crawling, e execução de código Python, enquanto retribui com a comunidade que o tornou possível.
Atualmente, o DeerFlow entrou oficialmente no Centro de Aplicações FaaS da Volcengine. Os usuários podem experimentá-lo online através do link de experiência para sentir intuitivamente suas funções poderosas e operações convenientes. Ao mesmo tempo, para atender às necessidades de implantação de diferentes usuários, o DeerFlow suporta implantação com um clique baseada na Volcengine. Clique no link de implantação para completar rapidamente o processo de implantação e iniciar uma jornada de pesquisa eficiente.
Por favor, visite [Nosso Site Oficial](https://deerflow.tech/) para maiores detalhes.
## Demo
### Video
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
Nesse demo, nós demonstramos como usar o DeerFlow para:
In this demo, we showcase how to use DeerFlow to:
- Integração fácil com serviços MCP
- Conduzir o processo de Pesquisa Profunda e produzir um relatório abrangente com imagens
- Criar um áudio podcast baseado no relatório gerado
### Replays
- [Quão alta é a Torre Eiffel comparada ao prédio mais alto?](https://deerflow.tech/chat?replay=eiffel-tower-vs-tallest-building)
- [Quais são os top repositórios tendência no GitHub?](https://deerflow.tech/chat?replay=github-top-trending-repo)
- [Escreva um artigo sobre os pratos tradicionais de Nanjing's](https://deerflow.tech/chat?replay=nanjing-traditional-dishes)
- [Como decorar um apartamento alugado?](https://deerflow.tech/chat?replay=rental-apartment-decoration)
- [Visite nosso site oficial para explorar mais replays.](https://deerflow.tech/#case-studies)
---
## 📑 Tabela de Conteúdos
- [🚀 Início Rápido](#Início-Rápido)
- [🌟 Funcionalidades](#funcionalidades)
- [🏗️ Arquitetura](#arquitetura)
- [🛠️ Desenvolvimento](#desenvolvimento)
- [🐳 Docker](#docker)
- [🗣️ Texto-para-fala Integração](#texto-para-fala-integração)
- [📚 Exemplos](#exemplos)
- [❓ FAQ](#faq)
- [📜 Licença](#licença)
- [💖 Agradecimentos](#agradecimentos)
- [🏆 Contribuidores-Chave](#contribuidores-chave)
- [⭐ Histórico de Estrelas](#Histórico-Estrelas)
## Início-Rápido
DeerFlow é desenvolvido em Python, e vem com uma IU web escrita em Node.js. Para garantir um processo de configuração fácil, nós recomendamos o uso das seguintes ferramentas:
### Ferramentas Recomendadas
- **[`uv`](https://docs.astral.sh/uv/getting-started/installation/):**
Simplifica o gerenciamento de dependência de ambientes Python. `uv` automaticamente cria um ambiente virtual no diretório raiz e instala todos os pacotes necessários para não haver a necessidade de instalar ambientes Python manualmente
- **[`nvm`](https://github.com/nvm-sh/nvm):**
Gerencia múltiplas versões do ambiente de execução do Node.js sem esforço.
- **[`pnpm`](https://pnpm.io/installation):**
Instala e gerencia dependências do projeto Node.js.
### Requisitos de Ambiente
Certifique-se de que seu sistema atenda os seguintes requisitos mínimos:
- **[Python](https://www.python.org/downloads/):** Versão `3.12+`
- **[Node.js](https://nodejs.org/en/download/):** Versão `22+`
### Instalação
```bash
# Clone o repositório
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
# Instale as dependências, uv irá lidar com o interpretador do python e a criação do venv, e instalar os pacotes necessários
uv sync
# Configure .env com suas chaves de API
# Tavily: https://app.tavily.com/home
# Brave_SEARCH: https://brave.com/search/api/
# volcengine TTS: Adicione sua credencial TTS caso você a possua
cp .env.example .env
# Veja as seções abaixo 'Supported Search Engines' and 'Texto-para-Fala Integração' para todas as opções disponíveis
# Configure o conf.yaml para o seu modelo LLM e chaves API
# Por favor, consulte 'docs/configuration_guide.md' para maiores detalhes
cp conf.yaml.example conf.yaml
# Instale marp para geração de ppt
# https://github.com/marp-team/marp-cli?tab=readme-ov-file#use-package-manager
brew install marp-cli
```
Opcionalmente, instale as dependências IU web via [pnpm](https://pnpm.io/installation):
```bash
cd deer-flow/web
pnpm install
```
### Configurações
Por favor, consulte o [Guia de Configuração](docs/configuration_guide.md) para maiores detalhes.
> [!NOTA]
> Antes de iniciar o projeto, leia o guia detalhadamente, e atualize as configurações para baterem com os seus requisitos e configurações específicas.
### Console IU
A maneira mais rápida de rodar o projeto é usar o console IU.
```bash
# Execute o projeto em um shell tipo-bash
uv run main.py
```
### Web IU
Esse projeto também inclui uma IU Web, trazendo uma experiência mais interativa, dinâmica e engajadora.
> [!NOTA]
> Você precisa instalar as dependências do IU web primeiro.
```bash
# Execute ambos os servidores de backend e frontend em modo desenvolvimento
# No macOS/Linux
./bootstrap.sh -d
# No Windows
bootstrap.bat -d
```
Abra seu navegador e visite [`http://localhost:3000`](http://localhost:3000) para explorar a IU web.
Explore mais detalhes no diretório [`web`](./web/) .
## Mecanismos de Busca Suportados
DeerFlow suporta múltiplos mecanismos de busca que podem ser configurados no seu arquivo `.env` usando a variável `SEARCH_API`:
- **Tavily** (padrão): Uma API de busca especializada para aplicações de IA
- Requer `TAVILY_API_KEY` no seu arquivo `.env`
- Inscreva-se em: <https://app.tavily.com/home>
- **DuckDuckGo**: Mecanismo de busca focado em privacidade
- Não requer chave API
- **Brave Search**: Mecanismo de busca focado em privacidade com funcionalidades avançadas
- Requer `BRAVE_SEARCH_API_KEY` no seu arquivo `.env`
- Inscreva-se em: <https://brave.com/search/api/>
- **Arxiv**: Busca de artigos científicos para pesquisa acadêmica
- Não requer chave API
- Especializado em artigos científicos e acadêmicos
Para configurar o seu mecanismo preferido, defina a variável `SEARCH_API` no seu arquivo:
```bash
# Escolha uma: tavily, duckduckgo, brave_search, arxiv
SEARCH_API=tavily
```
## Funcionalidades
### Principais Funcionalidades
- 🤖 **Integração LLM**
- Suporta a integração da maioria dos modelos através de [litellm](https://docs.litellm.ai/docs/providers).
- Suporte a modelos open source como Qwen
- Interface API compatível com a OpenAI
- Sistema LLM multicamadas para diferentes complexidades de tarefa
### Ferramentas e Integrações MCP
- 🔍 **Busca e Recuperação**
- Busca web com Tavily, Brave Search e mais
- Crawling com Jina
- Extração de Conteúdo avançada
- 🔗 **Integração MCP perfeita**
- Expansão de capacidades de acesso para acesso a domínios privados, grafo de conhecimento, navegação web e mais
- Integração facilitdade de diversas ferramentas de pesquisa e metodologias
### Colaboração Humana
- 🧠 **Humano-no-processo**
- Suporta modificação interativa de planos de pesquisa usando linguagem natural
- Suporta auto-aceite de planos de pesquisa
- 📝 **Relatório Pós-Edição**
- Suporta edição de edição de blocos estilo Notion
- Permite refinamentos de IA, incluindo polimento de IA assistida, encurtamento de frase, e expansão
- Distribuído por [tiptap](https://tiptap.dev/)
### Criação de Conteúdo
- 🎙️ **Geração de Podcast e apresentação**
- Script de geração de podcast e síntese de áudio movido por IA
- Criação automatizada de apresentações PowerPoint simples
- Templates customizáveis para conteúdo personalizado
## Arquitetura
DeerFlow implementa uma arquitetura de sistema multi-agente modular designada para pesquisa e análise de código automatizada. O sistema é construído em LangGraph, possibilitando um fluxo de trabalho flexível baseado-em-estado onde os componentes se comunicam através de um sistema de transmissão de mensagens bem-definido.
![Diagrama de Arquitetura](./assets/architecture.png)
> Veja ao vivo em [deerflow.tech](https://deerflow.tech/#multi-agent-architecture)
O sistema emprega um fluxo de trabalho simplificado com os seguintes componentes:
1. **Coordenador**: O ponto de entrada que gerencia o ciclo de vida do fluxo de trabalho
- Inicia o processo de pesquisa baseado na entrada do usuário
- Delega tarefas so planejador quando apropriado
- Atua como a interface primária entre o usuário e o sistema
2. **Planejador**: Componente estratégico para a decomposição e planejamento
- Analisa objetivos de pesquisa e cria planos de execução estruturados
- Determina se há contexto suficiente disponível ou se mais pesquisa é necessária
- Gerencia o fluxo de pesquisa e decide quando gerar o relatório final
3. **Time de Pesquisa**: Uma coleção de agentes especializados que executam o plano:
- **Pesquisador**: Conduz buscas web e coleta informações utilizando ferramentas como mecanismos de busca web, crawling e mesmo serviços MCP.
- **Programador**: Lida com a análise de código, execução e tarefas técnicas como usar a ferramenta Python REPL.
Cada agente tem acesso à ferramentas específicas otimizadas para seu papel e opera dentro do fluxo de trabalho LangGraph.
4. **Repórter**: Estágio final do processador de estágio para saídas de pesquisa
- Resultados agregados do time de pesquisa
- Processa e estrutura as informações coletadas
- Gera relatórios abrangentes de pesquisas
## Texto-para-Fala Integração
DeerFlow agora inclui uma funcionalidade Texto-para-Fala (TTS) que permite que você converta relatórios de busca para voz. Essa funcionalidade usa o mecanismo de voz da API TTS para gerar áudio de alta qualidade a partir do texto. Funcionalidades como velocidade, volume e tom também são customizáveis.
### Usando a API TTS
Você pode acessar a funcionalidade TTS através do endpoint `/api/tts`:
```bash
# Exemplo de chamada da API usando curl
curl --location 'http://localhost:8000/api/tts' \
--header 'Content-Type: application/json' \
--data '{
"text": "This is a test of the text-to-speech functionality.",
"speed_ratio": 1.0,
"volume_ratio": 1.0,
"pitch_ratio": 1.0
}' \
--output speech.mp3
```
## Desenvolvimento
### Testando
Rode o conjunto de testes:
```bash
# Roda todos os testes
make test
# Roda um arquivo de teste específico
pytest tests/integration/test_workflow.py
# Roda com coverage
make coverage
```
### Qualidade de Código
```bash
# Roda o linting
make lint
# Formata de código
make format
```
### Debugando com o LangGraph Studio
DeerFlow usa LangGraph para sua arquitetura de fluxo de trabalho. Nós podemos usar o LangGraph Studio para debugar e visualizar o fluxo de trabalho em tempo real.
#### Rodando o LangGraph Studio Localmente
DeerFlow inclui um arquivo de configuração `langgraph.json` que define a estrutura do grafo e dependências para o LangGraph Studio. Esse arquivo aponta para o grafo do fluxo de trabalho definido no projeto e automaticamente carrega as variáveis de ambiente do arquivo `.env`.
##### Mac
```bash
# Instala o gerenciador de pacote uv caso você não o possua
curl -LsSf https://astral.sh/uv/install.sh | sh
# Instala as dependências e inicia o servidor LangGraph
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.12 langgraph dev --allow-blocking
```
##### Windows / Linux
```bash
# Instala as dependências
pip install -e .
pip install -U "langgraph-cli[inmem]"
# Inicia o servidor LangGraph
langgraph dev
```
Após iniciar o servidor LangGraph, você verá diversas URLs no seu terminal:
- API: <http://127.0.0.1:2024>
- Studio UI: <https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024>
- API Docs: <http://127.0.0.1:2024/docs>
Abra o link do Studio UI no seu navegador para acessar a interface de depuração.
#### Usando o LangGraph Studio
No Studio UI, você pode:
1. Visualizar o grafo do fluxo de trabalho e como seus componentes se conectam
2. Rastrear a execução em tempo-real e ver como os dados fluem através do sistema
3. Inspecionar o estado de cada passo do fluxo de trabalho
4. Depurar problemas ao examinar entradas e saídas de cada componente
5. Coletar feedback durante a fase de planejamento para refinar os planos de pesquisa
Quando você envia um tópico de pesquisa ao Studio UI, você será capaz de ver toda a execução do fluxo de trabalho, incluindo:
- A fase de planejamento onde o plano de pesquisa foi criado
- O processo de feedback onde você pode modificar o plano
- As fases de pesquisa e escrita de cada seção
- A geração do relatório final
## Docker
Você também pode executar esse projeto via Docker.
Primeiro, voce deve ler a [configuração](#configuration) below. Make sure `.env`, `.conf.yaml` files are ready.
Segundo, para fazer o build de sua imagem docker em seu próprio servidor:
```bash
docker build -t deer-flow-api .
```
E por fim, inicie um container docker rodando o servidor web:
```bash
# substitua deer-flow-api-app com seu nome de container preferido
docker run -d -t -p 8000:8000 --env-file .env --name deer-flow-api-app deer-flow-api
# pare o servidor
docker stop deer-flow-api-app
```
### Docker Compose (inclui ambos backend e frontend)
DeerFlow fornece uma estrutura docker-compose para facilmente executar ambos o backend e frontend juntos:
```bash
# building docker image
docker compose build
# start the server
docker compose up
```
## Exemplos
Os seguintes exemplos demonstram as capacidades do DeerFlow:
### Relatórios de Pesquisa
1. **Relatório OpenAI Sora** - Análise da ferramenta Sora da OpenAI
- Discute funcionalidades, acesso, engenharia de prompt, limitações e considerações éticas
- [Veja o relatório completo](examples/openai_sora_report.md)
2. **Relatório Protocolo Agent-to-Agent do Google** - Visão geral do protocolo Agent-to-Agent (A2A) do Google
- Discute o seu papel na comunicação de Agente de IA e seu relacionamento com o Protocolo de Contexto de Modelo ( MCP ) da Anthropic
- [Veja o relatório completo](examples/what_is_agent_to_agent_protocol.md)
3. **O que é MCP?** - Uma análise abrangente to termo "MCP" através de múltiplos contextos
- Explora o Protocolo de Contexto de Modelo em IA, Fosfato Monocálcio em Química, e placa de microcanal em eletrônica
- [Veja o relatório completo](examples/what_is_mcp.md)
4. **Bitcoin Price Fluctuations** - Análise das recentes movimentações de preço do Bitcoin
- Examina tendências de mercado, influências regulatórias, e indicadores técnicos
- Fornece recomendações baseadas nos dados históricos
- [Veja o relatório completo](examples/bitcoin_price_fluctuation.md)
5. **O que é LLM?** - Uma exploração em profundidade de Large Language Models
- Discute arquitetura, treinamento, aplicações, e considerações éticas
- [Veja o relatório completo](examples/what_is_llm.md)
6. **Como usar Claude para Pesquisa Aprofundada?** - Melhores práticas e fluxos de trabalho para usar Claude em pesquisa aprofundada
- Cobre engenharia de prompt, análise de dados, e integração com outras ferramentas
- [Veja o relatório completo](examples/how_to_use_claude_deep_research.md)
7. **Adoção de IA na Área da Saúde: Fatores de Influência** - Análise dos fatores que levam à adoção de IA na área da saúde
- Discute tecnologias de IA, qualidade de dados, considerações éticas, avaliações econômicas, prontidão organizacional, e infraestrutura digital
- [Veja o relatório completo](examples/AI_adoption_in_healthcare.md)
8. **Impacto da Computação Quântica em Criptografia** - Análise dos impactos da computação quântica em criptografia
- Discture vulnerabilidades da criptografia clássica, criptografia pós-quântica, e soluções criptográficas de resistência-quântica
- [Veja o relatório completo](examples/Quantum_Computing_Impact_on_Cryptography.md)
9. **Destaques da Performance do Cristiano Ronaldo** - Análise dos destaques da performance do Cristiano Ronaldo
- Discute as suas conquistas de carreira, objetivos internacionais, e performance em diversas partidas
- [Veja o relatório completo](examples/Cristiano_Ronaldo's_Performance_Highlights.md)
Para executar esses exemplos ou criar seus próprios relatórios de pesquisa, você deve utilizar os seguintes comandos:
```bash
# Executa com uma consulta específica
uv run main.py "Quais fatores estão influenciando a adoção de IA na área da saúde?"
# Executa com parâmetros de planejamento customizados
uv run main.py --max_plan_iterations 3 "Como a computação quântica impacta na criptografia?"
# Executa em modo interativo com questões embutidas
uv run main.py --interactive
# Ou executa com um prompt interativo básico
uv run main.py
# Vê todas as opções disponíveis
uv run main.py --help
```
### Modo Interativo
A aplicação agora suporta um modo interativo com questões embutidas tanto em Inglês quanto Chinês:
1. Inicie o modo interativo:
```bash
uv run main.py --interactive
```
2. Selecione sua linguagem de preferência (English or 中文)
3. Escolha uma das questões embutidas da lista ou selecione a opção para perguntar sua própria questão
4. O sistema irá processar sua questão e gerar um relatório abrangente de pesquisa
### Humano no processo
DeerFlow inclue um mecanismo de humano no processo que permite a você revisar, editar e aprovar planos de pesquisa antes que estes sejam executados:
1. **Revisão de Plano**: Quando o humano no processo está habilitado, o sistema irá apresentar o plano de pesquisa gerado para sua revisão antes da execução
2. **Fornecimento de Feedback**: Você pode:
- Aceitar o plano respondendo com `[ACCEPTED]`
- Edite o plano fornecendo feedback (e.g., `[EDIT PLAN] Adicione mais passos sobre a implementação técnica`)
- O sistema irá incorporar seu feedback e gerar um plano revisado
3. **Auto-aceite**: Você pode habilitar o auto-aceite ou pular o processo de revisão:
- Via API: Defina `auto_accepted_plan: true` na sua requisição
4. **Integração de API**: Quanto usar a API, você pode fornecer um feedback através do parâmetro `feedback`:
```json
{
"messages": [{ "role": "user", "content": "O que é computação quântica?" }],
"thread_id": "my_thread_id",
"auto_accepted_plan": false,
"feedback": "[EDIT PLAN] Inclua mais sobre algoritmos quânticos"
}
```
### Argumentos via Linha de Comando
A aplicação suporta diversos argumentos via linha de comando para customizar o seu comportamento:
- **consulta**: A consulta de pesquisa a ser processada (podem ser múltiplas palavras)
- **--interativo**: Roda no modo interativo com questões embutidas
- **--max_plan_iterations**: Número máximo de ciclos de planejamento (padrão: 1)
- **--max_step_num**: Número máximo de passos em um plano de pesquisa (padrão: 3)
- **--debug**: Habilita Enable um log de depuração detalhado
## FAQ
Por favor consulte a [FAQ.md](docs/FAQ.md) para maiores detalhes.
## Licença
Esse projeto é open source e disponível sob a [MIT License](./LICENSE).
## Agradecimentos
DeerFlow é construído através do incrível trabalho da comunidade open-source. Nós somos profundamente gratos a todos os projetos e contribuidores cujos esforços tornaram o DeerFlow possível. Realmente, nós estamos apoiados nos ombros de gigantes.
Nós gostaríamos de extender nossos sinceros agradecimentos aos seguintes projetos por suas invaloráveis contribuições:
- **[LangChain](https://github.com/langchain-ai/langchain)**: O framework excepcional deles empodera nossas interações via LLM e correntes, permitindo uma integração perfeita e funcional.
- **[LangGraph](https://github.com/langchain-ai/langgraph)**: A abordagem inovativa para orquestração multi-agente deles tem sido foi fundamental em permitir o acesso dos fluxos de trabalho sofisticados do DeerFlow.
Esses projetos exemplificam o poder transformador da colaboração open-source, e nós temos orgulho de construir baseado em suas fundações.
### Contribuidores-Chave
Um sincero muito obrigado vai para os principais autores do `DeerFlow`, cuja visão, paixão, e dedicação trouxe esse projeto à vida:
- **[Daniel Walnut](https://github.com/hetaoBackend/)**
- **[Henry Li](https://github.com/magiccube/)**
O seu compromisso inabalável e experiência tem sido a força por trás do sucesso do DeerFlow. Nós estamos honrados em tê-los no comando dessa trajetória.
## Histórico-Estrelas
[![Gráfico do Histórico de Estrelas](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
+559
View File
@@ -0,0 +1,559 @@
# 🦌 DeerFlow
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DeepWiki](https://img.shields.io/badge/DeepWiki-bytedance%2Fdeer--flow-blue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McCcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==)](https://deepwiki.com/bytedance/deer-flow)
<!-- DeepWiki badge generated by https://deepwiki.ryoppippi.com/ -->
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) | [Portuguese](./README_pt.md)
> Создано на базе открытого кода, возвращено в открытый код.
**DeerFlow** (**D**eep **E**xploration and **E**fficient **R**esearch **Flow**) - это фреймворк для глубокого исследования, разработанный сообществом и основанный на впечатляющей работе сообщества открытого кода. Наша цель - объединить языковые модели со специализированными инструментами для таких задач, как веб-поиск, сканирование и выполнение кода Python, одновременно возвращая пользу сообществу, которое сделало это возможным.
В настоящее время DeerFlow официально вошел в Центр приложений FaaS Volcengine. Пользователи могут испытать его онлайн через ссылку для опыта, чтобы интуитивно почувствовать его мощные функции и удобные операции. В то же время, для удовлетворения потребностей развертывания различных пользователей, DeerFlow поддерживает развертывание одним кликом на основе Volcengine. Нажмите на ссылку развертывания, чтобы быстро завершить процесс развертывания и начать эффективное исследовательское путешествие.
Пожалуйста, посетите [наш официальный сайт](https://deerflow.tech/) для получения дополнительной информации.
## Демонстрация
### Видео
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
В этой демонстрации мы показываем, как использовать DeerFlow для:
- Бесшовной интеграции с сервисами MCP
- Проведения процесса глубокого исследования и создания комплексного отчета с изображениями
- Создания аудио подкаста на основе сгенерированного отчета
### Повторы
- [Какова высота Эйфелевой башни по сравнению с самым высоким зданием?](https://deerflow.tech/chat?replay=eiffel-tower-vs-tallest-building)
- [Какие репозитории самые популярные на GitHub?](https://deerflow.tech/chat?replay=github-top-trending-repo)
- [Написать статью о традиционных блюдах Нанкина](https://deerflow.tech/chat?replay=nanjing-traditional-dishes)
- [Как украсить съемную квартиру?](https://deerflow.tech/chat?replay=rental-apartment-decoration)
- [Посетите наш официальный сайт, чтобы изучить больше повторов.](https://deerflow.tech/#case-studies)
---
## 📑 Оглавление
- [🚀 Быстрый старт](#быстрый-старт)
- [🌟 Особенности](#особенности)
- [🏗️ Архитектура](#архитектура)
- [🛠️ Разработка](#разработка)
- [🐳 Docker](#docker)
- [🗣️ Интеграция преобразования текста в речь](#интеграция-преобразования-текста-в-речь)
- [📚 Примеры](#примеры)
- [❓ FAQ](#faq)
- [📜 Лицензия](#лицензия)
- [💖 Благодарности](#благодарности)
- [⭐ История звезд](#история-звезд)
## Быстрый старт
DeerFlow разработан на Python и поставляется с веб-интерфейсом, написанным на Node.js. Для обеспечения плавного процесса настройки мы рекомендуем использовать следующие инструменты:
### Рекомендуемые инструменты
- **[`uv`](https://docs.astral.sh/uv/getting-started/installation/):**
Упрощает управление средой Python и зависимостями. `uv` автоматически создает виртуальную среду в корневом каталоге и устанавливает все необходимые пакеты за вас—без необходимости вручную устанавливать среды Python.
- **[`nvm`](https://github.com/nvm-sh/nvm):**
Легко управляйте несколькими версиями среды выполнения Node.js.
- **[`pnpm`](https://pnpm.io/installation):**
Установка и управление зависимостями проекта Node.js.
### Требования к среде
Убедитесь, что ваша система соответствует следующим минимальным требованиям:
- **[Python](https://www.python.org/downloads/):** Версия `3.12+`
- **[Node.js](https://nodejs.org/en/download/):** Версия `22+`
### Установка
```bash
# Клонировать репозиторий
git clone https://github.com/bytedance/deer-flow.git
cd deer-flow
# Установить зависимости, uv позаботится об интерпретаторе python и создании venv, и установит необходимые пакеты
uv sync
# Настроить .env с вашими API-ключами
# Tavily: https://app.tavily.com/home
# Brave_SEARCH: https://brave.com/search/api/
# volcengine TTS: Добавьте ваши учетные данные TTS, если они у вас есть
cp .env.example .env
# См. разделы 'Поддерживаемые поисковые системы' и 'Интеграция преобразования текста в речь' ниже для всех доступных опций
# Настроить conf.yaml для вашей модели LLM и API-ключей
# Пожалуйста, обратитесь к 'docs/configuration_guide.md' для получения дополнительной информации
cp conf.yaml.example conf.yaml
# Установить marp для генерации презентаций
# https://github.com/marp-team/marp-cli?tab=readme-ov-file#use-package-manager
brew install marp-cli
```
По желанию установите зависимости веб-интерфейса через [pnpm](https://pnpm.io/installation):
```bash
cd deer-flow/web
pnpm install
```
### Конфигурации
Пожалуйста, обратитесь к [Руководству по конфигурации](docs/configuration_guide.md) для получения дополнительной информации.
> [!ПРИМЕЧАНИЕ]
> Прежде чем запустить проект, внимательно прочитайте руководство и обновите конфигурации в соответствии с вашими конкретными настройками и требованиями.
### Консольный интерфейс
Самый быстрый способ запустить проект - использовать консольный интерфейс.
```bash
# Запустить проект в оболочке, похожей на bash
uv run main.py
```
### Веб-интерфейс
Этот проект также включает веб-интерфейс, предлагающий более динамичный и привлекательный интерактивный опыт.
> [!ПРИМЕЧАНИЕ]
> Сначала вам нужно установить зависимости веб-интерфейса.
```bash
# Запустить оба сервера, бэкенд и фронтенд, в режиме разработки
# На macOS/Linux
./bootstrap.sh -d
# На Windows
bootstrap.bat -d
```
Откройте ваш браузер и посетите [`http://localhost:3000`](http://localhost:3000), чтобы исследовать веб-интерфейс.
Исследуйте больше деталей в каталоге [`web`](./web/).
## Поддерживаемые поисковые системы
DeerFlow поддерживает несколько поисковых систем, которые можно настроить в файле `.env` с помощью переменной `SEARCH_API`:
- **Tavily** (по умолчанию): Специализированный поисковый API для приложений ИИ
- Требуется `TAVILY_API_KEY` в вашем файле `.env`
- Зарегистрируйтесь на: <https://app.tavily.com/home>
- **DuckDuckGo**: Поисковая система, ориентированная на конфиденциальность
- Не требуется API-ключ
- **Brave Search**: Поисковая система, ориентированная на конфиденциальность, с расширенными функциями
- Требуется `BRAVE_SEARCH_API_KEY` в вашем файле `.env`
- Зарегистрируйтесь на: <https://brave.com/search/api/>
- **Arxiv**: Поиск научных статей для академических исследований
- Не требуется API-ключ
- Специализируется на научных и академических статьях
Чтобы настроить предпочитаемую поисковую систему, установите переменную `SEARCH_API` в вашем файле `.env`:
```bash
# Выберите одно: tavily, duckduckgo, brave_search, arxiv
SEARCH_API=tavily
```
## Особенности
### Ключевые возможности
- 🤖 **Интеграция LLM**
- Поддерживает интеграцию большинства моделей через [litellm](https://docs.litellm.ai/docs/providers).
- Поддержка моделей с открытым исходным кодом, таких как Qwen
- API-интерфейс, совместимый с OpenAI
- Многоуровневая система LLM для задач различной сложности
### Инструменты и интеграции MCP
- 🔍 **Поиск и извлечение**
- Веб-поиск через Tavily, Brave Search и другие
- Сканирование с Jina
- Расширенное извлечение контента
- 🔗 **Бесшовная интеграция MCP**
- Расширение возможностей для доступа к частным доменам, графам знаний, веб-браузингу и многому другому
- Облегчает интеграцию различных исследовательских инструментов и методологий
### Человеческое взаимодействие
- 🧠 **Человек в контуре**
- Поддерживает интерактивное изменение планов исследования с использованием естественного языка
- Поддерживает автоматическое принятие планов исследования
- 📝 **Пост-редактирование отчетов**
- Поддерживает блочное редактирование в стиле Notion
- Позволяет совершенствовать с помощью ИИ, включая полировку, сокращение и расширение предложений
- Работает на [tiptap](https://tiptap.dev/)
### Создание контента
- 🎙️ **Генерация подкастов и презентаций**
- Генерация сценариев подкастов и синтез аудио с помощью ИИ
- Автоматическое создание простых презентаций PowerPoint
- Настраиваемые шаблоны для индивидуального контента
## Архитектура
DeerFlow реализует модульную архитектуру системы с несколькими агентами, предназначенную для автоматизированных исследований и анализа кода. Система построена на LangGraph, обеспечивающей гибкий рабочий процесс на основе состояний, где компоненты взаимодействуют через четко определенную систему передачи сообщений.
![Диаграмма архитектуры](./assets/architecture.png)
> Посмотрите вживую на [deerflow.tech](https://deerflow.tech/#multi-agent-architecture)
В системе используется оптимизированный рабочий процесс со следующими компонентами:
1. **Координатор**: Точка входа, управляющая жизненным циклом рабочего процесса
- Инициирует процесс исследования на основе пользовательского ввода
- Делегирует задачи планировщику, когда это необходимо
- Выступает в качестве основного интерфейса между пользователем и системой
2. **Планировщик**: Стратегический компонент для декомпозиции и планирования задач
- Анализирует цели исследования и создает структурированные планы выполнения
- Определяет, достаточно ли доступного контекста или требуется дополнительное исследование
- Управляет потоком исследования и решает, когда генерировать итоговый отчет
3. **Исследовательская команда**: Набор специализированных агентов, которые выполняют план:
- **Исследователь**: Проводит веб-поиск и сбор информации с использованием таких инструментов, как поисковые системы, сканирование и даже сервисы MCP.
- **Программист**: Обрабатывает анализ кода, выполнение и технические задачи с помощью инструмента Python REPL.
Каждый агент имеет доступ к определенным инструментам, оптимизированным для его роли, и работает в рамках фреймворка LangGraph
4. **Репортер**: Процессор финальной стадии для результатов исследования
- Агрегирует находки исследовательской команды
- Обрабатывает и структурирует собранную информацию
- Генерирует комплексные исследовательские отчеты
## Интеграция преобразования текста в речь
DeerFlow теперь включает функцию преобразования текста в речь (TTS), которая позволяет конвертировать исследовательские отчеты в речь. Эта функция использует API TTS volcengine для генерации высококачественного аудио из текста. Также можно настраивать такие параметры, как скорость, громкость и тон.
### Использование API TTS
Вы можете получить доступ к функциональности TTS через конечную точку `/api/tts`:
```bash
# Пример вызова API с использованием curl
curl --location 'http://localhost:8000/api/tts' \
--header 'Content-Type: application/json' \
--data '{
"text": "Это тест функциональности преобразования текста в речь.",
"speed_ratio": 1.0,
"volume_ratio": 1.0,
"pitch_ratio": 1.0
}' \
--output speech.mp3
```
## Разработка
### Тестирование
Запустите набор тестов:
```bash
# Запустить все тесты
make test
# Запустить определенный тестовый файл
pytest tests/integration/test_workflow.py
# Запустить с покрытием
make coverage
```
### Качество кода
```bash
# Запустить линтинг
make lint
# Форматировать код
make format
```
### Отладка с LangGraph Studio
DeerFlow использует LangGraph для своей архитектуры рабочего процесса. Вы можете использовать LangGraph Studio для отладки и визуализации рабочего процесса в реальном времени.
#### Запуск LangGraph Studio локально
DeerFlow включает конфигурационный файл `langgraph.json`, который определяет структуру графа и зависимости для LangGraph Studio. Этот файл указывает на графы рабочего процесса, определенные в проекте, и автоматически загружает переменные окружения из файла `.env`.
##### Mac
```bash
# Установите менеджер пакетов uv, если у вас его нет
curl -LsSf https://astral.sh/uv/install.sh | sh
# Установите зависимости и запустите сервер LangGraph
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.12 langgraph dev --allow-blocking
```
##### Windows / Linux
```bash
# Установить зависимости
pip install -e .
pip install -U "langgraph-cli[inmem]"
# Запустить сервер LangGraph
langgraph dev
```
После запуска сервера LangGraph вы увидите несколько URL в терминале:
- API: <http://127.0.0.1:2024>
- Studio UI: <https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024>
- API Docs: <http://127.0.0.1:2024/docs>
Откройте ссылку Studio UI в вашем браузере для доступа к интерфейсу отладки.
#### Использование LangGraph Studio
В интерфейсе Studio вы можете:
1. Визуализировать граф рабочего процесса и видеть, как соединяются компоненты
2. Отслеживать выполнение в реальном времени, чтобы видеть, как данные проходят через систему
3. Исследовать состояние на каждом шаге рабочего процесса
4. Отлаживать проблемы путем изучения входов и выходов каждого компонента
5. Предоставлять обратную связь во время фазы планирования для уточнения планов исследования
Когда вы отправляете тему исследования в интерфейсе Studio, вы сможете увидеть весь процесс выполнения рабочего процесса, включая:
- Фазу планирования, где создается план исследования
- Цикл обратной связи, где вы можете модифицировать план
- Фазы исследования и написания для каждого раздела
- Генерацию итогового отчета
### Включение трассировки LangSmith
DeerFlow поддерживает трассировку LangSmith, чтобы помочь вам отладить и контролировать ваши рабочие процессы. Чтобы включить трассировку LangSmith:
1. Убедитесь, что в вашем файле `.env` есть следующие конфигурации (см. `.env.example`):
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
LANGSMITH_API_KEY="xxx"
LANGSMITH_PROJECT="xxx"
```
2. Запустите трассировку и визуализируйте граф локально с LangSmith, выполнив:
```bash
langgraph dev
```
Это включит визуализацию трассировки в LangGraph Studio и отправит ваши трассировки в LangSmith для мониторинга и анализа.
## Docker
Вы также можете запустить этот проект с Docker.
Во-первых, вам нужно прочитать [конфигурацию](docs/configuration_guide.md) ниже. Убедитесь, что файлы `.env`, `.conf.yaml` готовы.
Во-вторых, чтобы построить Docker-образ вашего собственного веб-сервера:
```bash
docker build -t deer-flow-api .
```
Наконец, запустите Docker-контейнер с веб-сервером:
```bash
# Замените deer-flow-api-app на предпочитаемое вами имя контейнера
docker run -d -t -p 8000:8000 --env-file .env --name deer-flow-api-app deer-flow-api
# остановить сервер
docker stop deer-flow-api-app
```
### Docker Compose (включает как бэкенд, так и фронтенд)
DeerFlow предоставляет настройку docker-compose для легкого запуска бэкенда и фронтенда вместе:
```bash
# сборка docker-образа
docker compose build
# запуск сервера
docker compose up
```
## Примеры
Следующие примеры демонстрируют возможности DeerFlow:
### Исследовательские отчеты
1. **Отчет о OpenAI Sora** - Анализ инструмента ИИ Sora от OpenAI
- Обсуждаются функции, доступ, инженерия промптов, ограничения и этические соображения
- [Просмотреть полный отчет](examples/openai_sora_report.md)
2. **Отчет о протоколе Agent to Agent от Google** - Обзор протокола Agent to Agent (A2A) от Google
- Обсуждается его роль в коммуникации агентов ИИ и его отношение к протоколу Model Context Protocol (MCP) от Anthropic
- [Просмотреть полный отчет](examples/what_is_agent_to_agent_protocol.md)
3. **Что такое MCP?** - Комплексный анализ термина "MCP" в различных контекстах
- Исследует Model Context Protocol в ИИ, Монокальцийфосфат в химии и Микроканальные пластины в электронике
- [Просмотреть полный отчет](examples/what_is_mcp.md)
4. **Колебания цены Биткоина** - Анализ недавних движений цены Биткоина
- Исследует рыночные тренды, регуляторные влияния и технические индикаторы
- Предоставляет рекомендации на основе исторических данных
- [Просмотреть полный отчет](examples/bitcoin_price_fluctuation.md)
5. **Что такое LLM?** - Углубленное исследование больших языковых моделей
- Обсуждаются архитектура, обучение, приложения и этические соображения
- [Просмотреть полный отчет](examples/what_is_llm.md)
6. **Как использовать Claude для глубокого исследования?** - Лучшие практики и рабочие процессы для использования Claude в глубоком исследовании
- Охватывает инженерию промптов, анализ данных и интеграцию с другими инструментами
- [Просмотреть полный отчет](examples/how_to_use_claude_deep_research.md)
7. **Внедрение ИИ в здравоохранении: Влияющие факторы** - Анализ факторов, движущих внедрением ИИ в здравоохранении
- Обсуждаются технологии ИИ, качество данных, этические соображения, экономические оценки, организационная готовность и цифровая инфраструктура
- [Просмотреть полный отчет](examples/AI_adoption_in_healthcare.md)
8. **Влияние квантовых вычислений на криптографию** - Анализ влияния квантовых вычислений на криптографию
- Обсуждаются уязвимости классической криптографии, пост-квантовая криптография и криптографические решения, устойчивые к квантовым вычислениям
- [Просмотреть полный отчет](examples/Quantum_Computing_Impact_on_Cryptography.md)
9. **Ключевые моменты выступлений Криштиану Роналду** - Анализ выдающихся выступлений Криштиану Роналду
- Обсуждаются его карьерные достижения, международные голы и выступления в различных матчах
- [Просмотреть полный отчет](examples/Cristiano_Ronaldo's_Performance_Highlights.md)
Чтобы запустить эти примеры или создать собственные исследовательские отчеты, вы можете использовать следующие команды:
```bash
# Запустить с определенным запросом
uv run main.py "Какие факторы влияют на внедрение ИИ в здравоохранении?"
# Запустить с пользовательскими параметрами планирования
uv run main.py --max_plan_iterations 3 "Как квантовые вычисления влияют на криптографию?"
# Запустить в интерактивном режиме с встроенными вопросами
uv run main.py --interactive
# Или запустить с базовым интерактивным приглашением
uv run main.py
# Посмотреть все доступные опции
uv run main.py --help
```
### Интерактивный режим
Приложение теперь поддерживает интерактивный режим с встроенными вопросами как на английском, так и на китайском языках:
1. Запустите интерактивный режим:
```bash
uv run main.py --interactive
```
2. Выберите предпочитаемый язык (English или 中文)
3. Выберите из списка встроенных вопросов или выберите опцию задать собственный вопрос
4. Система обработает ваш вопрос и сгенерирует комплексный исследовательский отчет
### Человек в контуре
DeerFlow включает механизм "человек в контуре", который позволяет вам просматривать, редактировать и утверждать планы исследования перед их выполнением:
1. **Просмотр плана**: Когда активирован режим "человек в контуре", система представит сгенерированный план исследования для вашего просмотра перед выполнением
2. **Предоставление обратной связи**: Вы можете:
- Принять план, ответив `[ACCEPTED]`
- Отредактировать план, предоставив обратную связь (например, `[EDIT PLAN] Добавить больше шагов о технической реализации`)
- Система включит вашу обратную связь и сгенерирует пересмотренный план
3. **Автоматическое принятие**: Вы можете включить автоматическое принятие, чтобы пропустить процесс просмотра:
- Через API: Установите `auto_accepted_plan: true` в вашем запросе
4. **Интеграция API**: При использовании API вы можете предоставить обратную связь через параметр `feedback`:
```json
{
"messages": [{ "role": "user", "content": "Что такое квантовые вычисления?" }],
"thread_id": "my_thread_id",
"auto_accepted_plan": false,
"feedback": "[EDIT PLAN] Включить больше о квантовых алгоритмах"
}
```
### Аргументы командной строки
Приложение поддерживает несколько аргументов командной строки для настройки его поведения:
- **query**: Запрос исследования для обработки (может состоять из нескольких слов)
- **--interactive**: Запустить в интерактивном режиме с встроенными вопросами
- **--max_plan_iterations**: Максимальное количество циклов планирования (по умолчанию: 1)
- **--max_step_num**: Максимальное количество шагов в плане исследования (по умолчанию: 3)
- **--debug**: Включить подробное логирование отладки
## FAQ
Пожалуйста, обратитесь к [FAQ.md](docs/FAQ.md) для получения дополнительной информации.
## Лицензия
Этот проект имеет открытый исходный код и доступен под [Лицензией MIT](./LICENSE).
## Благодарности
DeerFlow создан на основе невероятной работы сообщества открытого кода. Мы глубоко благодарны всем проектам и контрибьюторам, чьи усилия сделали DeerFlow возможным. Поистине, мы стоим на плечах гигантов.
Мы хотели бы выразить искреннюю признательность следующим проектам за их неоценимый вклад:
- **[LangChain](https://github.com/langchain-ai/langchain)**: Их исключительный фреймворк обеспечивает наши взаимодействия и цепочки LLM, позволяя бесшовную интеграцию и функциональность.
- **[LangGraph](https://github.com/langchain-ai/langgraph)**: Их инновационный подход к оркестровке многоагентных систем сыграл решающую роль в обеспечении сложных рабочих процессов DeerFlow.
Эти проекты являются примером преобразующей силы сотрудничества в области открытого кода, и мы гордимся тем, что строим на их основе.
### Ключевые контрибьюторы
Сердечная благодарность основным авторам `DeerFlow`, чье видение, страсть и преданность делу вдохнули жизнь в этот проект:
- **[Daniel Walnut](https://github.com/hetaoBackend/)**
- **[Henry Li](https://github.com/magiccube/)**
Ваша непоколебимая приверженность и опыт стали движущей силой успеха DeerFlow. Мы считаем за честь иметь вас во главе этого путешествия.
## История звезд
[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/deer-flow&type=Date)](https://star-history.com/#bytedance/deer-flow&Date)
+71 -28
View File
@@ -3,19 +3,21 @@
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md)
[English](./README.md) | [简体中文](./README_zh.md) | [日本語](./README_ja.md) | [Deutsch](./README_de.md) | [Español](./README_es.md) | [Русский](./README_ru.md) |[Portuguese](./README_pt.md)
> 源于开源,回馈开源。
**DeerFlow****D**eep **E**xploration and **E**fficient **R**esearch **Flow**)是一个社区驱动的深度研究框架,它建立在开源社区的杰出工作基础之上。我们的目标是将语言模型与专业工具(如网络搜索、爬虫和 Python 代码执行)相结合,同时回馈使这一切成为可能的社区。
目前,DeerFlow 已正式入驻[火山引擎的 FaaS 应用中心](https://console.volcengine.com/vefaas/region:vefaas+cn-beijing/market),用户可通过[体验链接](https://console.volcengine.com/vefaas/region:vefaas+cn-beijing/market/deerflow/?channel=github&source=deerflow)进行在线体验,直观感受其强大功能与便捷操作;同时,为满足不同用户的部署需求,DeerFlow 支持基于火山引擎一键部署,点击[部署链接](https://console.volcengine.com/vefaas/region:vefaas+cn-beijing/application/create?templateId=683adf9e372daa0008aaed5c&channel=github&source=deerflow)即可快速完成部署流程,开启高效研究之旅。
请访问[DeerFlow 的官方网站](https://deerflow.tech/)了解更多详情。
## 演示
### 视频
https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e
<https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e>
在此演示中,我们展示了如何使用 DeerFlow:
@@ -44,7 +46,7 @@ https://github.com/user-attachments/assets/f3786598-1f2a-4d07-919e-8b99dfa1de3e
- [❓ 常见问题](#常见问题)
- [📜 许可证](#许可证)
- [💖 致谢](#致谢)
- [⭐ Star History](#star-History)
- [⭐ Star History](#star-history)
## 快速开始
@@ -106,7 +108,7 @@ pnpm install
请参阅[配置指南](docs/configuration_guide.md)获取更多详情。
> [!注意]
> [! 注意]
> 在启动项目之前,请仔细阅读指南,并更新配置以匹配您的特定设置和要求。
### 控制台 UI
@@ -121,8 +123,7 @@ uv run main.py
### Web UI
本项目还包括一个 Web UI,提供更加动态和引人入胜的交互体验。
> [!注意]
> [! 注意]
> 您需要先安装 Web UI 的依赖。
```bash
@@ -140,21 +141,20 @@ bootstrap.bat -d
## 支持的搜索引擎
### 公域搜索引擎
DeerFlow 支持多种搜索引擎,可以在`.env`文件中通过`SEARCH_API`变量进行配置:
- **Tavily**(默认):专为 AI 应用设计的专业搜索 API
- 需要在`.env`文件中设置`TAVILY_API_KEY`
- 注册地址:https://app.tavily.com/home
- 注册地址:<https://app.tavily.com/home>
- **DuckDuckGo**:注重隐私的搜索引擎
- 无需 API 密钥
- **Brave Search**:具有高级功能的注重隐私的搜索引擎
- 需要在`.env`文件中设置`BRAVE_SEARCH_API_KEY`
- 注册地址:https://brave.com/search/api/
- 注册地址:<https://brave.com/search/api/>
- **Arxiv**:用于学术研究的科学论文搜索
- 无需 API 密钥
@@ -167,6 +167,30 @@ DeerFlow 支持多种搜索引擎,可以在`.env`文件中通过`SEARCH_API`
SEARCH_API=tavily
```
### 私域知识库引擎
DeerFlow 支持基于私有域知识的检索,您可以将文档上传到多种私有知识库中,以便在研究过程中使用,当前支持的私域知识库有:
- **[RAGFlow](https://ragflow.io/docs/dev/)**:开源的基于检索增强生成的知识库引擎
```
# 参照示例进行配置 .env.example
RAG_PROVIDER=ragflow
RAGFLOW_API_URL="http://localhost:9388"
RAGFLOW_API_KEY="ragflow-xxx"
RAGFLOW_RETRIEVAL_SIZE=10
```
- **[VikingDB 知识库](https://www.volcengine.com/docs/84313/1254457)**:火山引擎提供的公有云知识库引擎
> 注意先从 [火山引擎](https://www.volcengine.com/docs/84313/1254485) 获取账号 AK/SK
```
# 参照示例进行配置 .env.example
RAG_PROVIDER=vikingdb_knowledge_base
VIKINGDB_KNOWLEDGE_BASE_API_URL="api-knowledgebase.mlp.cn-beijing.volces.com"
VIKINGDB_KNOWLEDGE_BASE_API_AK="volcengine-ak-xxx"
VIKINGDB_KNOWLEDGE_BASE_API_SK="volcengine-sk-xxx"
VIKINGDB_KNOWLEDGE_BASE_RETRIEVAL_SIZE=15
```
## 特性
### 核心能力
@@ -180,10 +204,14 @@ SEARCH_API=tavily
### 工具和 MCP 集成
- 🔍 **搜索和检索**
- 通过 Tavily、Brave Search 等进行网络搜索
- 使用 Jina 进行爬取
- 高级内容提取
- 支持检索指定私有知识库
- 📃 **RAG 集成**
- 支持 [RAGFlow](https://github.com/infiniflow/ragflow) 知识库
- 支持 [VikingDB](https://www.volcengine.com/docs/84313/1254457) 火山知识库
- 🔗 **MCP 无缝集成**
- 扩展私有域访问、知识图谱、网页浏览等能力
@@ -192,7 +220,6 @@ SEARCH_API=tavily
### 人机协作
- 🧠 **人在环中**
- 支持使用自然语言交互式修改研究计划
- 支持自动接受研究计划
@@ -231,16 +258,36 @@ DeerFlow 实现了一个模块化的多智能体系统架构,专为自动化
- 管理研究流程并决定何时生成最终报告
3. **研究团队**:执行计划的专业智能体集合:
- **研究员**:使用网络搜索引擎、爬虫甚至 MCP 服务等工具进行网络搜索和信息收集。
- **编码员**:使用 Python REPL 工具处理代码分析、执行和技术任务。
每个智能体都可以访问针对其角色优化的特定工具,并在 LangGraph 框架内运行
每个智能体都可以访问针对其角色优化的特定工具,并在 LangGraph 框架内运行
4. **报告员**:研究输出的最终阶段处理器
- 汇总研究团队的发现
- 处理和组织收集的信息
- 生成全面的研究报告
## 文本转语音集成
DeerFlow 现在包含一个文本转语音 (TTS) 功能,允许您将研究报告转换为语音。此功能使用火山引擎 TTS API 生成高质量的文本音频。速度、音量和音调等特性也可以自定义。
### 使用 TTS API
您可以通过`/api/tts`端点访问 TTS 功能:
```bash
# 使用curl的API调用示例
curl --location 'http://localhost:8000/api/tts' \
--header 'Content-Type: application/json' \
--data '{
"text": "这是文本转语音功能的测试。",
"speed_ratio": 1.0,
"volume_ratio": 1.0,
"pitch_ratio": 1.0
}' \
--output speech.mp3
```
## 开发
### 测试
@@ -299,9 +346,9 @@ langgraph dev
启动 LangGraph 服务器后,您将在终端中看到几个 URL:
- API: http://127.0.0.1:2024
- Studio UI: https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024
- API 文档: http://127.0.0.1:2024/docs
- API: <http://127.0.0.1:2024>
- Studio UI: <https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024>
- API 文档<http://127.0.0.1:2024/docs>
在浏览器中打开 Studio UI 链接以访问调试界面。
@@ -327,6 +374,7 @@ langgraph dev
DeerFlow 支持 LangSmith 追踪功能,帮助您调试和监控工作流。要启用 LangSmith 追踪:
1. 确保您的 `.env` 文件中有以下配置(参见 `.env.example`):
```bash
LANGSMITH_TRACING=true
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
@@ -335,6 +383,7 @@ DeerFlow 支持 LangSmith 追踪功能,帮助您调试和监控工作流。要
```
2. 通过运行以下命令本地启动 LangSmith 追踪:
```bash
langgraph dev
```
@@ -377,7 +426,7 @@ docker compose up
## 文本转语音集成
DeerFlow 现在包含一个文本转语音(TTS)功能,允许您将研究报告转换为语音。此功能使用火山引擎 TTS API 生成高质量的文本音频。速度、音量和音调等特性也可以自定义。
DeerFlow 现在包含一个文本转语音 (TTS) 功能,允许您将研究报告转换为语音。此功能使用火山引擎 TTS API 生成高质量的文本音频。速度、音量和音调等特性也可以自定义。
### 使用 TTS API
@@ -403,17 +452,14 @@ curl --location 'http://localhost:8000/api/tts' \
### 研究报告
1. **OpenAI Sora 报告** - OpenAI 的 Sora AI 工具分析
- 讨论功能、访问方式、提示工程、限制和伦理考虑
- [查看完整报告](examples/openai_sora_report.md)
2. **Google 的 Agent to Agent 协议报告** - Google 的 Agent to Agent (A2A)协议概述
- 讨论其在 AI 智能体通信中的作用及其与 Anthropic 的 Model Context Protocol (MCP)的关系
2. **Google 的 Agent to Agent 协议报告** - Google 的 Agent to Agent (A2A) 协议概述
- 讨论其在 AI 智能体通信中的作用及其与 Anthropic 的 Model Context Protocol (MCP) 的关系
- [查看完整报告](examples/what_is_agent_to_agent_protocol.md)
3. **什么是 MCP** - 对"MCP"一词在多个上下文中的全面分析
- 探讨 AI 中的 Model Context Protocol、化学中的 Monocalcium Phosphate 和电子学中的 Micro-channel Plate
- [查看完整报告](examples/what_is_mcp.md)
@@ -424,17 +470,14 @@ curl --location 'http://localhost:8000/api/tts' \
- [查看完整报告](examples/bitcoin_price_fluctuation.md)
5. **什么是 LLM?** - 对大型语言模型的深入探索
- 讨论架构、训练、应用和伦理考虑
- [查看完整报告](examples/what_is_llm.md)
6. **如何使用 Claude 进行深度研究?** - 在深度研究中使用 Claude 的最佳实践和工作流程
- 涵盖提示工程、数据分析和与其他工具的集成
- [查看完整报告](examples/how_to_use_claude_deep_research.md)
7. **医疗保健中的 AI 采用:影响因素** - 影响医疗保健中 AI 采用的因素分析
- 讨论 AI 技术、数据质量、伦理考虑、经济评估、组织准备度和数字基础设施
- [查看完整报告](examples/AI_adoption_in_healthcare.md)
@@ -495,10 +538,10 @@ DeerFlow 包含一个人在环中机制,允许您在执行研究计划前审
- 系统将整合您的反馈并生成修订后的计划
3. **自动接受**:您可以启用自动接受以跳过审查过程:
- 通过 API:在请求中设置`auto_accepted_plan: true`
4. **API 集成**:使用 API 时,您可以通过`feedback`参数提供反馈:
```json
{
"messages": [{ "role": "user", "content": "什么是量子计算?" }],
+3 -2
View File
@@ -10,9 +10,10 @@ IF "%MODE%"=="development" GOTO DEV
:PROD
echo Starting DeerFlow in [PRODUCTION] mode...
uv run server.py
start uv run server.py
cd web
pnpm start
start pnpm start
REM Wait for user to close
GOTO END
:DEV
+4 -2
View File
@@ -11,6 +11,8 @@ if [ "$1" = "--dev" -o "$1" = "-d" -o "$1" = "dev" -o "$1" = "development" ]; th
wait
else
echo -e "Starting DeerFlow in [PRODUCTION] mode...\n"
uv run server.py
cd web && pnpm start
uv run server.py & SERVER_PID=$$!
cd web && pnpm start & WEB_PID=$$!
trap "kill $$SERVER_PID $$WEB_PID" SIGINT SIGTERM
wait
fi
+34 -3
View File
@@ -1,9 +1,40 @@
# [!NOTE]
# Read the `docs/configuration_guide.md` carefully, and update the configurations to match your specific settings and requirements.
# - Replace `api_key` with your own credentials
# - Replace `base_url` and `model` name if you want to use a custom model
# Read the `docs/configuration_guide.md` carefully, and update the
# configurations to match your specific settings and requirements.
# - Replace `api_key` with your own credentials.
# - Replace `base_url` and `model` name if you want to use a custom model.
# - Set `verify_ssl` to `false` if your LLM server uses self-signed certificates
# - A restart is required every time you change the `config.yaml` file.
BASIC_MODEL:
base_url: https://ark.cn-beijing.volces.com/api/v3
model: "doubao-1-5-pro-32k-250115"
api_key: xxxx
# max_retries: 3 # Maximum number of retries for LLM calls
# verify_ssl: false # Uncomment this line to disable SSL certificate verification for self-signed certificates
# Reasoning model is optional.
# Uncomment the following settings if you want to use reasoning model
# for planning.
# REASONING_MODEL:
# base_url: https://ark.cn-beijing.volces.com/api/v3
# model: "doubao-1-5-thinking-pro-m-250428"
# api_key: xxxx
# max_retries: 3 # Maximum number of retries for LLM calls
# OTHER SETTINGS:
# Search engine configuration (Only supports Tavily currently)
# SEARCH_ENGINE:
# engine: tavily
# # Only include results from these domains
# include_domains:
# - example.com
# - trusted-news.com
# - reliable-source.org
# - gov.cn
# - edu.cn
# # Exclude results from these domains
# exclude_domains:
# - example.com
+1 -1
View File
@@ -9,7 +9,7 @@ services:
env_file:
- .env
volumes:
- ./conf.yaml:/app/conf.yaml
- ./conf.yaml:/app/conf.yaml:ro
restart: unless-stopped
networks:
- deer-flow-network
+53 -12
View File
@@ -11,7 +11,7 @@ cp conf.yaml.example conf.yaml
## Which models does DeerFlow support?
In DeerFlow, currently we only support non-reasoning models, which means models like OpenAI's o1/o3 or DeepSeek's R1 are not supported yet, but we will add support for them in the future.
In DeerFlow, we currently only support non-reasoning models. This means models like OpenAI's o1/o3 or DeepSeek's R1 are not supported yet, but we plan to add support for them in the future. Additionally, all Gemma-3 models are currently unsupported due to the lack of tool usage capabilities.
### Supported Models
@@ -49,7 +49,7 @@ BASIC_MODEL:
BASIC_MODEL:
base_url: "https://api.deepseek.com"
model: "deepseek-chat"
api_key: YOU_API_KEY
api_key: YOUR_API_KEY
# An example of Google Gemini models using OpenAI-Compatible interface
BASIC_MODEL:
@@ -58,15 +58,31 @@ BASIC_MODEL:
api_key: YOUR_API_KEY
```
### How to use Ollama models?
### How to use models with self-signed SSL certificates?
DeerFlow supports the integration of Ollama models. You can refer to [litellm Ollama](https://docs.litellm.ai/docs/providers/ollama). <br>
The following is a configuration example of `conf.yaml` for using Ollama models:
If your LLM server uses self-signed SSL certificates, you can disable SSL certificate verification by adding the `verify_ssl: false` parameter to your model configuration:
```yaml
BASIC_MODEL:
model: "ollama/ollama-model-name"
base_url: "http://localhost:11434" # Local service address of Ollama, which can be started/viewed via ollama serve
base_url: "https://your-llm-server.com/api/v1"
model: "your-model-name"
api_key: YOUR_API_KEY
verify_ssl: false # Disable SSL certificate verification for self-signed certificates
```
> [!WARNING]
> Disabling SSL certificate verification reduces security and should only be used in development environments or when you trust the LLM server. In production environments, it's recommended to use properly signed SSL certificates.
### How to use Ollama models?
DeerFlow supports the integration of Ollama models. You can refer to [litellm Ollama](https://docs.litellm.ai/docs/providers/ollama). <br>
The following is a configuration example of `conf.yaml` for using Ollama models(you might need to run the 'ollama serve' first):
```yaml
BASIC_MODEL:
model: "model-name" # Model name, which supports the completions API(important), such as: qwen3:8b, mistral-small3.1:24b, qwen2.5:3b
base_url: "http://localhost:11434/v1" # Local service address of Ollama, which can be started/viewed via ollama serve
api_key: "whatever" # Mandatory, fake api_key with a random string you like :-)
```
### How to use OpenRouter models?
@@ -89,13 +105,38 @@ BASIC_MODEL:
Note: The available models and their exact names may change over time. Please verify the currently available models and their correct identifiers in [OpenRouter's official documentation](https://openrouter.ai/docs).
### How to use Azure models?
DeerFlow supports the integration of Azure models. You can refer to [litellm Azure](https://docs.litellm.ai/docs/providers/azure). Configuration example of `conf.yaml`:
### How to use Azure OpenAI chat models?
DeerFlow supports the integration of Azure OpenAI chat models. You can refer to [AzureChatOpenAI](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html). Configuration example of `conf.yaml`:
```yaml
BASIC_MODEL:
model: "azure/gpt-4o-2024-08-06"
api_base: $AZURE_API_BASE
api_version: $AZURE_API_VERSION
api_key: $AZURE_API_KEY
azure_endpoint: $AZURE_OPENAI_ENDPOINT
api_version: $OPENAI_API_VERSION
api_key: $AZURE_OPENAI_API_KEY
```
## About Search Engine
### How to control search domains for Tavily?
DeerFlow allows you to control which domains are included or excluded in Tavily search results through the configuration file. This helps improve search result quality and reduce hallucinations by focusing on trusted sources.
`Tips`: it only supports Tavily currently.
You can configure domain filtering in your `conf.yaml` file as follows:
```yaml
SEARCH_ENGINE:
engine: tavily
# Only include results from these domains (whitelist)
include_domains:
- trusted-news.com
- gov.org
- reliable-source.edu
# Exclude results from these domains (blacklist)
exclude_domains:
- unreliable-site.com
- spam-domain.net
+10 -1
View File
@@ -1,4 +1,13 @@
# MCP Integrations
# MCP IntegrationsBeta
Now This feature is diabled by default. You can enable it by setting the environment ENABLE_MCP_SERVER_CONFIGURATION to be true
> [!WARNING]
> Please enable this feature before securing your frond-end and back-end in a managed environment.
> Otherwise, you system could be compromised.
This feature is diabled by default. You can enable it by setting the environment ENABLE_MCP_SERVER_CONFIGURATION
Please enable this feature before securing your frond-end and back-end in an internal environment.q
## Example of MCP Server Configuration
+5 -1
View File
@@ -140,7 +140,11 @@ if __name__ == "__main__":
if args.query:
user_query = " ".join(args.query)
else:
user_query = input("Enter your query: ")
# Loop until user provides non-empty input
while True:
user_query = input("Enter your query: ")
if user_query is not None and user_query != "":
break
# Run the agent workflow with the provided parameters
ask(
+11
View File
@@ -32,17 +32,25 @@ dependencies = [
"arxiv>=2.2.0",
"mcp>=1.6.0",
"langchain-mcp-adapters>=0.0.9",
"langchain-deepseek>=0.1.3",
"volcengine>=1.0.191",
]
[project.optional-dependencies]
dev = [
"ruff",
"black>=24.2.0",
"langgraph-cli[inmem]>=0.2.10",
]
test = [
"pytest>=7.4.0",
"pytest-cov>=4.1.0",
"pytest-asyncio>=1.0.0",
]
[tool.uv]
required-version = ">=0.6.15"
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
@@ -52,6 +60,9 @@ filterwarnings = [
"ignore::UserWarning",
]
[tool.coverage.report]
fail_under = 25
[tool.hatch.build.targets.wheel]
packages = ["src"]
+25 -11
View File
@@ -7,7 +7,8 @@ Server script for running the DeerFlow API.
import argparse
import logging
import signal
import sys
import uvicorn
# Configure logging
@@ -18,6 +19,17 @@ logging.basicConfig(
logger = logging.getLogger(__name__)
def handle_shutdown(signum, frame):
"""Handle graceful shutdown on SIGTERM/SIGINT"""
logger.info("Received shutdown signal. Starting graceful shutdown...")
sys.exit(0)
# Register signal handlers
signal.signal(signal.SIGTERM, handle_shutdown)
signal.signal(signal.SIGINT, handle_shutdown)
if __name__ == "__main__":
# Parse command line arguments
parser = argparse.ArgumentParser(description="Run the DeerFlow API server")
@@ -50,16 +62,18 @@ if __name__ == "__main__":
# Determine reload setting
reload = False
# Command line arguments override defaults
if args.reload:
reload = True
logger.info("Starting DeerFlow API server")
uvicorn.run(
"src.server:app",
host=args.host,
port=args.port,
reload=reload,
log_level=args.log_level,
)
try:
logger.info(f"Starting DeerFlow API server on {args.host}:{args.port}")
uvicorn.run(
"src.server:app",
host=args.host,
port=args.port,
reload=reload,
log_level=args.log_level,
)
except Exception as e:
logger.error(f"Failed to start server: {str(e)}")
sys.exit(1)
+2 -2
View File
@@ -1,6 +1,6 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from .agents import research_agent, coder_agent
from .agents import create_agent
__all__ = ["research_agent", "coder_agent"]
__all__ = ["create_agent"]
-13
View File
@@ -4,12 +4,6 @@
from langgraph.prebuilt import create_react_agent
from src.prompts import apply_prompt_template
from src.tools import (
crawl_tool,
python_repl_tool,
web_search_tool,
)
from src.llms.llm import get_llm_by_type
from src.config.agents import AGENT_LLM_MAP
@@ -23,10 +17,3 @@ def create_agent(agent_name: str, agent_type: str, tools: list, prompt_template:
tools=tools,
prompt=lambda state: apply_prompt_template(prompt_template, state),
)
# Create agents using the factory function
research_agent = create_agent(
"researcher", "researcher", [web_search_tool, crawl_tool], "researcher"
)
coder_agent = create_agent("coder", "coder", [python_repl_tool], "coder")
+5 -5
View File
@@ -1,8 +1,8 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from .tools import SEARCH_MAX_RESULTS, SELECTED_SEARCH_ENGINE, SearchEngine
from .loader import load_yaml_config
from .tools import SELECTED_SEARCH_ENGINE, SearchEngine
from .questions import BUILT_IN_QUESTIONS, BUILT_IN_QUESTIONS_ZH_CN
from dotenv import load_dotenv
@@ -11,7 +11,7 @@ from dotenv import load_dotenv
load_dotenv()
# Team configuration
TEAM_MEMBER_CONFIGRATIONS = {
TEAM_MEMBER_CONFIGURATIONS = {
"researcher": {
"name": "researcher",
"desc": (
@@ -36,15 +36,15 @@ TEAM_MEMBER_CONFIGRATIONS = {
},
}
TEAM_MEMBERS = list(TEAM_MEMBER_CONFIGRATIONS.keys())
TEAM_MEMBERS = list(TEAM_MEMBER_CONFIGURATIONS.keys())
__all__ = [
# Other configurations
"TEAM_MEMBERS",
"TEAM_MEMBER_CONFIGRATIONS",
"SEARCH_MAX_RESULTS",
"TEAM_MEMBER_CONFIGURATIONS",
"SELECTED_SEARCH_ENGINE",
"SearchEngine",
"BUILT_IN_QUESTIONS",
"BUILT_IN_QUESTIONS_ZH_CN",
load_yaml_config,
]
+1
View File
@@ -16,4 +16,5 @@ AGENT_LLM_MAP: dict[str, LLMType] = {
"podcast_script_writer": "basic",
"ppt_composer": "basic",
"prose_writer": "basic",
"prompt_enhancer": "basic",
}
+10 -1
View File
@@ -2,19 +2,28 @@
# SPDX-License-Identifier: MIT
import os
from dataclasses import dataclass, fields
from dataclasses import dataclass, field, fields
from typing import Any, Optional
from langchain_core.runnables import RunnableConfig
from src.rag.retriever import Resource
from src.config.report_style import ReportStyle
@dataclass(kw_only=True)
class Configuration:
"""The configurable fields."""
resources: list[Resource] = field(
default_factory=list
) # Resources to be used for the research
max_plan_iterations: int = 1 # Maximum number of plan iterations
max_step_num: int = 3 # Maximum number of steps in a plan
max_search_results: int = 3 # Maximum number of search results
mcp_settings: dict = None # MCP settings, including dynamic loaded tools
report_style: str = ReportStyle.ACADEMIC.value # Report style
enable_deep_thinking: bool = False # Whether to enable deep thinking
@classmethod
def from_runnable_config(
+3 -1
View File
@@ -12,12 +12,14 @@ def replace_env_vars(value: str) -> str:
return value
if value.startswith("$"):
env_var = value[1:]
return os.getenv(env_var, value)
return os.getenv(env_var, env_var)
return value
def process_dict(config: Dict[str, Any]) -> Dict[str, Any]:
"""Recursively process dictionary to replace environment variables."""
if not config:
return {}
result = {}
for key, value in config.items():
if isinstance(value, dict):
+8
View File
@@ -0,0 +1,8 @@
import enum
class ReportStyle(enum.Enum):
ACADEMIC = "academic"
POPULAR_SCIENCE = "popular_science"
NEWS = "news"
SOCIAL_MEDIA = "social_media"
+8 -1
View File
@@ -17,4 +17,11 @@ class SearchEngine(enum.Enum):
# Tool configuration
SELECTED_SEARCH_ENGINE = os.getenv("SEARCH_API", SearchEngine.TAVILY.value)
SEARCH_MAX_RESULTS = 3
class RAGProvider(enum.Enum):
RAGFLOW = "ragflow"
VIKINGDB_KNOWLEDGE_BASE = "vikingdb_knowledge_base"
SELECTED_RAG_PROVIDER = os.getenv("RAG_PROVIDER")
+3 -4
View File
@@ -3,8 +3,7 @@
from .article import Article
from .crawler import Crawler
from .jina_client import JinaClient
from .readability_extractor import ReadabilityExtractor
__all__ = [
"Article",
"Crawler",
]
__all__ = ["Article", "Crawler", "JinaClient", "ReadabilityExtractor"]
-11
View File
@@ -1,7 +1,6 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import sys
from .article import Article
from .jina_client import JinaClient
@@ -26,13 +25,3 @@ class Crawler:
article = extractor.extract_article(html)
article.url = url
return article
if __name__ == "__main__":
if len(sys.argv) == 2:
url = sys.argv[1]
else:
url = "https://fintel.io/zh-hant/s/br/nvdc34"
crawler = Crawler()
article = crawler.crawl(url)
print(article.to_markdown())
+32
View File
@@ -3,6 +3,7 @@
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.memory import MemorySaver
from src.prompts.planner_model import StepType
from .types import State
from .nodes import (
@@ -17,6 +18,31 @@ from .nodes import (
)
def continue_to_running_research_team(state: State):
current_plan = state.get("current_plan")
if not current_plan or not current_plan.steps:
return "planner"
if all(step.execution_res for step in current_plan.steps):
return "planner"
# Find first incomplete step
incomplete_step = None
for step in current_plan.steps:
if not step.execution_res:
incomplete_step = step
break
if not incomplete_step:
return "planner"
if incomplete_step.step_type == StepType.RESEARCH:
return "researcher"
if incomplete_step.step_type == StepType.PROCESSING:
return "coder"
return "planner"
def _build_base_graph():
"""Build and return the base state graph with all nodes and edges."""
builder = StateGraph(State)
@@ -29,6 +55,12 @@ def _build_base_graph():
builder.add_node("researcher", researcher_node)
builder.add_node("coder", coder_node)
builder.add_node("human_feedback", human_feedback_node)
builder.add_edge("background_investigator", "planner")
builder.add_conditional_edges(
"research_team",
continue_to_running_research_team,
["planner", "researcher", "coder"],
)
builder.add_edge("reporter", END)
return builder
+114 -70
View File
@@ -3,6 +3,7 @@
import json
import logging
import os
from typing import Annotated, Literal
from langchain_core.messages import AIMessage, HumanMessage
@@ -11,31 +12,31 @@ from langchain_core.tools import tool
from langgraph.types import Command, interrupt
from langchain_mcp_adapters.client import MultiServerMCPClient
from src.agents.agents import coder_agent, research_agent, create_agent
from src.agents import create_agent
from src.tools.search import LoggedTavilySearch
from src.tools import (
crawl_tool,
web_search_tool,
get_web_search_tool,
get_retriever_tool,
python_repl_tool,
)
from src.config.agents import AGENT_LLM_MAP
from src.config.configuration import Configuration
from src.llms.llm import get_llm_by_type
from src.prompts.planner_model import Plan, StepType
from src.prompts.planner_model import Plan
from src.prompts.template import apply_prompt_template
from src.utils.json_utils import repair_json_output
from .types import State
from ..config import SEARCH_MAX_RESULTS, SELECTED_SEARCH_ENGINE, SearchEngine
from ..config import SELECTED_SEARCH_ENGINE, SearchEngine
logger = logging.getLogger(__name__)
@tool
def handoff_to_planner(
task_title: Annotated[str, "The title of the task to be handed off."],
research_topic: Annotated[str, "The topic of the research task to be handed off."],
locale: Annotated[str, "The user's detected language locale (e.g., en-US, zh-CN)."],
):
"""Handoff to planner agent to do plan."""
@@ -44,33 +45,37 @@ def handoff_to_planner(
return
def background_investigation_node(state: State) -> Command[Literal["planner"]]:
def background_investigation_node(state: State, config: RunnableConfig):
logger.info("background investigation node is running.")
query = state["messages"][-1].content
if SELECTED_SEARCH_ENGINE == SearchEngine.TAVILY:
searched_content = LoggedTavilySearch(max_results=SEARCH_MAX_RESULTS).invoke(
{"query": query}
)
background_investigation_results = None
configurable = Configuration.from_runnable_config(config)
query = state.get("research_topic")
background_investigation_results = None
if SELECTED_SEARCH_ENGINE == SearchEngine.TAVILY.value:
searched_content = LoggedTavilySearch(
max_results=configurable.max_search_results
).invoke(query)
if isinstance(searched_content, list):
background_investigation_results = [
{"title": elem["title"], "content": elem["content"]}
for elem in searched_content
f"## {elem['title']}\n\n{elem['content']}" for elem in searched_content
]
return {
"background_investigation_results": "\n\n".join(
background_investigation_results
)
}
else:
logger.error(
f"Tavily search returned malformed response: {searched_content}"
)
else:
background_investigation_results = web_search_tool.invoke(query)
return Command(
update={
"background_investigation_results": json.dumps(
background_investigation_results, ensure_ascii=False
)
},
goto="planner",
)
background_investigation_results = get_web_search_tool(
configurable.max_search_results
).invoke(query)
return {
"background_investigation_results": json.dumps(
background_investigation_results, ensure_ascii=False
)
}
def planner_node(
@@ -82,10 +87,8 @@ def planner_node(
plan_iterations = state["plan_iterations"] if state.get("plan_iterations", 0) else 0
messages = apply_prompt_template("planner", state, configurable)
if (
plan_iterations == 0
and state.get("enable_background_investigation")
and state.get("background_investigation_results")
if state.get("enable_background_investigation") and state.get(
"background_investigation_results"
):
messages += [
{
@@ -98,8 +101,10 @@ def planner_node(
}
]
if AGENT_LLM_MAP["planner"] == "basic":
llm = get_llm_by_type(AGENT_LLM_MAP["planner"]).with_structured_output(
if configurable.enable_deep_thinking:
llm = get_llm_by_type("reasoning")
elif AGENT_LLM_MAP["planner"] == "basic":
llm = get_llm_by_type("basic").with_structured_output(
Plan,
method="json_mode",
)
@@ -111,7 +116,7 @@ def planner_node(
return Command(goto="reporter")
full_response = ""
if AGENT_LLM_MAP["planner"] == "basic":
if AGENT_LLM_MAP["planner"] == "basic" and not configurable.enable_deep_thinking:
response = llm.invoke(messages)
full_response = response.model_dump_json(indent=4, exclude_none=True)
else:
@@ -129,7 +134,7 @@ def planner_node(
return Command(goto="reporter")
else:
return Command(goto="__end__")
if curr_plan.get("has_enough_context"):
if isinstance(curr_plan, dict) and curr_plan.get("has_enough_context"):
logger.info("Planner response has enough context.")
new_plan = Plan.model_validate(curr_plan)
return Command(
@@ -181,11 +186,9 @@ def human_feedback_node(
plan_iterations += 1
# parse the plan
new_plan = json.loads(current_plan)
if new_plan["has_enough_context"]:
goto = "reporter"
except json.JSONDecodeError:
logger.warning("Planner response is not a valid JSON")
if plan_iterations > 0:
if plan_iterations > 1: # the plan_iterations is increased before this check
return Command(goto="reporter")
else:
return Command(goto="__end__")
@@ -201,10 +204,11 @@ def human_feedback_node(
def coordinator_node(
state: State,
state: State, config: RunnableConfig
) -> Command[Literal["planner", "background_investigator", "__end__"]]:
"""Coordinator node that communicate with customers."""
logger.info("Coordinator talking.")
configurable = Configuration.from_runnable_config(config)
messages = apply_prompt_template("coordinator", state)
response = (
get_llm_by_type(AGENT_LLM_MAP["coordinator"])
@@ -215,6 +219,7 @@ def coordinator_node(
goto = "__end__"
locale = state.get("locale", "en-US") # Default locale if not specified
research_topic = state.get("research_topic", "")
if len(response.tool_calls) > 0:
goto = "planner"
@@ -225,8 +230,11 @@ def coordinator_node(
for tool_call in response.tool_calls:
if tool_call.get("name", "") != "handoff_to_planner":
continue
if tool_locale := tool_call.get("args", {}).get("locale"):
locale = tool_locale
if tool_call.get("args", {}).get("locale") and tool_call.get(
"args", {}
).get("research_topic"):
locale = tool_call.get("args", {}).get("locale")
research_topic = tool_call.get("args", {}).get("research_topic")
break
except Exception as e:
logger.error(f"Error processing tool calls: {e}")
@@ -235,16 +243,24 @@ def coordinator_node(
"Coordinator response contains no tool calls. Terminating workflow execution."
)
logger.debug(f"Coordinator response: {response}")
messages = state.get("messages", [])
if response.content:
messages.append(HumanMessage(content=response.content, name="coordinator"))
return Command(
update={"locale": locale},
update={
"messages": messages,
"locale": locale,
"research_topic": research_topic,
"resources": configurable.resources,
},
goto=goto,
)
def reporter_node(state: State):
def reporter_node(state: State, config: RunnableConfig):
"""Reporter node that write a final report."""
logger.info("Reporter write final report")
configurable = Configuration.from_runnable_config(config)
current_plan = state.get("current_plan")
input_ = {
"messages": [
@@ -254,7 +270,7 @@ def reporter_node(state: State):
],
"locale": state.get("locale", "en-US"),
}
invoke_messages = apply_prompt_template("reporter", input_)
invoke_messages = apply_prompt_template("reporter", input_, configurable)
observations = state.get("observations", [])
# Add a reminder about the new report format, citation style, and table usage
@@ -280,24 +296,10 @@ def reporter_node(state: State):
return {"final_report": response_content}
def research_team_node(
state: State,
) -> Command[Literal["planner", "researcher", "coder"]]:
def research_team_node(state: State):
"""Research team node that collaborates on tasks."""
logger.info("Research team is collaborating on tasks.")
current_plan = state.get("current_plan")
if not current_plan or not current_plan.steps:
return Command(goto="planner")
if all(step.execution_res for step in current_plan.steps):
return Command(goto="planner")
for step in current_plan.steps:
if not step.execution_res:
break
if step.step_type and step.step_type == StepType.RESEARCH:
return Command(goto="researcher")
if step.step_type and step.step_type == StepType.PROCESSING:
return Command(goto="coder")
return Command(goto="planner")
pass
async def _execute_agent_step(
@@ -305,6 +307,7 @@ async def _execute_agent_step(
) -> Command[Literal["research_team"]]:
"""Helper function to execute a step using the specified agent."""
current_plan = state.get("current_plan")
plan_title = current_plan.title
observations = state.get("observations", [])
# Find the first unexecuted step
@@ -321,27 +324,40 @@ async def _execute_agent_step(
logger.warning("No unexecuted step found")
return Command(goto="research_team")
logger.info(f"Executing step: {current_step.title}")
logger.info(f"Executing step: {current_step.title}, agent: {agent_name}")
# Format completed steps information
completed_steps_info = ""
if completed_steps:
completed_steps_info = "# Existing Research Findings\n\n"
completed_steps_info = "# Completed Research Steps\n\n"
for i, step in enumerate(completed_steps):
completed_steps_info += f"## Existing Finding {i+1}: {step.title}\n\n"
completed_steps_info += f"## Completed Step {i + 1}: {step.title}\n\n"
completed_steps_info += f"<finding>\n{step.execution_res}\n</finding>\n\n"
# Prepare the input for the agent with completed steps info
agent_input = {
"messages": [
HumanMessage(
content=f"{completed_steps_info}# Current Task\n\n## Title\n\n{current_step.title}\n\n## Description\n\n{current_step.description}\n\n## Locale\n\n{state.get('locale', 'en-US')}"
content=f"# Research Topic\n\n{plan_title}\n\n{completed_steps_info}# Current Step\n\n## Title\n\n{current_step.title}\n\n## Description\n\n{current_step.description}\n\n## Locale\n\n{state.get('locale', 'en-US')}"
)
]
}
# Add citation reminder for researcher agent
if agent_name == "researcher":
if state.get("resources"):
resources_info = "**The user mentioned the following resource files:**\n\n"
for resource in state.get("resources"):
resources_info += f"- {resource.title} ({resource.description})\n"
agent_input["messages"].append(
HumanMessage(
content=resources_info
+ "\n\n"
+ "You MUST use the **local_search_tool** to retrieve the information from the resource files.",
)
)
agent_input["messages"].append(
HumanMessage(
content="IMPORTANT: DO NOT include inline citations in the text. Instead, track all sources and include a References section at the end using link reference format. Include an empty line between each citation for better readability. Use this format for each reference:\n- [Source Title](URL)\n\n- [Another Source](URL)",
@@ -350,7 +366,32 @@ async def _execute_agent_step(
)
# Invoke the agent
result = await agent.ainvoke(input=agent_input)
default_recursion_limit = 25
try:
env_value_str = os.getenv("AGENT_RECURSION_LIMIT", str(default_recursion_limit))
parsed_limit = int(env_value_str)
if parsed_limit > 0:
recursion_limit = parsed_limit
logger.info(f"Recursion limit set to: {recursion_limit}")
else:
logger.warning(
f"AGENT_RECURSION_LIMIT value '{env_value_str}' (parsed as {parsed_limit}) is not positive. "
f"Using default value {default_recursion_limit}."
)
recursion_limit = default_recursion_limit
except ValueError:
raw_env_value = os.getenv("AGENT_RECURSION_LIMIT")
logger.warning(
f"Invalid AGENT_RECURSION_LIMIT value: '{raw_env_value}'. "
f"Using default value {default_recursion_limit}."
)
recursion_limit = default_recursion_limit
logger.info(f"Agent input: {agent_input}")
result = await agent.ainvoke(
input=agent_input, config={"recursion_limit": recursion_limit}
)
# Process the result
response_content = result["messages"][-1].content
@@ -378,7 +419,6 @@ async def _setup_and_execute_agent_step(
state: State,
config: RunnableConfig,
agent_type: str,
default_agent,
default_tools: list,
) -> Command[Literal["research_team"]]:
"""Helper function to set up an agent with appropriate tools and execute a step.
@@ -392,7 +432,6 @@ async def _setup_and_execute_agent_step(
state: The current state
config: The runnable config
agent_type: The type of agent ("researcher" or "coder")
default_agent: The default agent to use if no MCP servers are configured
default_tools: The default tools to add to the agent
Returns:
@@ -430,8 +469,9 @@ async def _setup_and_execute_agent_step(
agent = create_agent(agent_type, agent_type, loaded_tools, agent_type)
return await _execute_agent_step(state, agent, agent_type)
else:
# Use default agent if no MCP servers are configured
return await _execute_agent_step(state, default_agent, agent_type)
# Use default tools if no MCP servers are configured
agent = create_agent(agent_type, agent_type, default_tools, agent_type)
return await _execute_agent_step(state, agent, agent_type)
async def researcher_node(
@@ -439,12 +479,17 @@ async def researcher_node(
) -> Command[Literal["research_team"]]:
"""Researcher node that do research"""
logger.info("Researcher node is researching.")
configurable = Configuration.from_runnable_config(config)
tools = [get_web_search_tool(configurable.max_search_results), crawl_tool]
retriever_tool = get_retriever_tool(state.get("resources", []))
if retriever_tool:
tools.insert(0, retriever_tool)
logger.info(f"Researcher tools: {tools}")
return await _setup_and_execute_agent_step(
state,
config,
"researcher",
research_agent,
[web_search_tool, crawl_tool],
tools,
)
@@ -457,6 +502,5 @@ async def coder_node(
state,
config,
"coder",
coder_agent,
[python_repl_tool],
)
+3 -2
View File
@@ -1,12 +1,11 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import operator
from typing import Annotated
from langgraph.graph import MessagesState
from src.prompts.planner_model import Plan
from src.rag import Resource
class State(MessagesState):
@@ -14,7 +13,9 @@ class State(MessagesState):
# Runtime Variables
locale: str = "en-US"
research_topic: str = ""
observations: list[str] = []
resources: list[Resource] = []
plan_iterations: int = 0
current_plan: Plan | str = None
final_report: str = ""
+115 -21
View File
@@ -3,54 +3,148 @@
from pathlib import Path
from typing import Any, Dict
import os
import httpx
from langchain_openai import ChatOpenAI
from langchain_core.language_models import BaseChatModel
from langchain_openai import ChatOpenAI, AzureChatOpenAI
from langchain_deepseek import ChatDeepSeek
from typing import get_args
from src.config import load_yaml_config
from src.config.agents import LLMType
# Cache for LLM instances
_llm_cache: dict[LLMType, ChatOpenAI] = {}
_llm_cache: dict[LLMType, BaseChatModel] = {}
def _create_llm_use_conf(llm_type: LLMType, conf: Dict[str, Any]) -> ChatOpenAI:
llm_type_map = {
"reasoning": conf.get("REASONING_MODEL"),
"basic": conf.get("BASIC_MODEL"),
"vision": conf.get("VISION_MODEL"),
def _get_config_file_path() -> str:
"""Get the path to the configuration file."""
return str((Path(__file__).parent.parent.parent / "conf.yaml").resolve())
def _get_llm_type_config_keys() -> dict[str, str]:
"""Get mapping of LLM types to their configuration keys."""
return {
"reasoning": "REASONING_MODEL",
"basic": "BASIC_MODEL",
"vision": "VISION_MODEL",
}
llm_conf = llm_type_map.get(llm_type)
if not llm_conf:
def _get_env_llm_conf(llm_type: str) -> Dict[str, Any]:
"""
Get LLM configuration from environment variables.
Environment variables should follow the format: {LLM_TYPE}__{KEY}
e.g., BASIC_MODEL__api_key, BASIC_MODEL__base_url
"""
prefix = f"{llm_type.upper()}_MODEL__"
conf = {}
for key, value in os.environ.items():
if key.startswith(prefix):
conf_key = key[len(prefix) :].lower()
conf[conf_key] = value
return conf
def _create_llm_use_conf(llm_type: LLMType, conf: Dict[str, Any]) -> BaseChatModel:
"""Create LLM instance using configuration."""
llm_type_config_keys = _get_llm_type_config_keys()
config_key = llm_type_config_keys.get(llm_type)
if not config_key:
raise ValueError(f"Unknown LLM type: {llm_type}")
llm_conf = conf.get(config_key, {})
if not isinstance(llm_conf, dict):
raise ValueError(f"Invalid LLM Conf: {llm_type}")
return ChatOpenAI(**llm_conf)
raise ValueError(f"Invalid LLM configuration for {llm_type}: {llm_conf}")
# Get configuration from environment variables
env_conf = _get_env_llm_conf(llm_type)
# Merge configurations, with environment variables taking precedence
merged_conf = {**llm_conf, **env_conf}
if not merged_conf:
raise ValueError(f"No configuration found for LLM type: {llm_type}")
# Add max_retries to handle rate limit errors
if "max_retries" not in merged_conf:
merged_conf["max_retries"] = 3
if llm_type == "reasoning":
merged_conf["api_base"] = merged_conf.pop("base_url", None)
# Handle SSL verification settings
verify_ssl = merged_conf.pop("verify_ssl", True)
# Create custom HTTP client if SSL verification is disabled
if not verify_ssl:
http_client = httpx.Client(verify=False)
http_async_client = httpx.AsyncClient(verify=False)
merged_conf["http_client"] = http_client
merged_conf["http_async_client"] = http_async_client
if "azure_endpoint" in merged_conf or os.getenv("AZURE_OPENAI_ENDPOINT"):
return AzureChatOpenAI(**merged_conf)
if llm_type == "reasoning":
return ChatDeepSeek(**merged_conf)
else:
return ChatOpenAI(**merged_conf)
def get_llm_by_type(
llm_type: LLMType,
) -> ChatOpenAI:
) -> BaseChatModel:
"""
Get LLM instance by type. Returns cached instance if available.
"""
if llm_type in _llm_cache:
return _llm_cache[llm_type]
conf = load_yaml_config(
str((Path(__file__).parent.parent.parent / "conf.yaml").resolve())
)
conf = load_yaml_config(_get_config_file_path())
llm = _create_llm_use_conf(llm_type, conf)
_llm_cache[llm_type] = llm
return llm
# Initialize LLMs for different purposes - now these will be cached
basic_llm = get_llm_by_type("basic")
def get_configured_llm_models() -> dict[str, list[str]]:
"""
Get all configured LLM models grouped by type.
Returns:
Dictionary mapping LLM type to list of configured model names.
"""
try:
conf = load_yaml_config(_get_config_file_path())
llm_type_config_keys = _get_llm_type_config_keys()
configured_models: dict[str, list[str]] = {}
for llm_type in get_args(LLMType):
# Get configuration from YAML file
config_key = llm_type_config_keys.get(llm_type, "")
yaml_conf = conf.get(config_key, {}) if config_key else {}
# Get configuration from environment variables
env_conf = _get_env_llm_conf(llm_type)
# Merge configurations, with environment variables taking precedence
merged_conf = {**yaml_conf, **env_conf}
# Check if model is configured
model_name = merged_conf.get("model")
if model_name:
configured_models.setdefault(llm_type, []).append(model_name)
return configured_models
except Exception as e:
# Log error and return empty dict to avoid breaking the application
print(f"Warning: Failed to load LLM configuration: {e}")
return {}
# In the future, we will use reasoning_llm and vl_llm for different purposes
# reasoning_llm = get_llm_by_type("reasoning")
# vl_llm = get_llm_by_type("vision")
if __name__ == "__main__":
print(basic_llm.invoke("Hello"))
-1
View File
@@ -1,7 +1,6 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from typing import Optional
from langgraph.graph import MessagesState
+4
View File
@@ -0,0 +1,4 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
"""Prompt enhancer module for improving user prompts."""
+25
View File
@@ -0,0 +1,25 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from langgraph.graph import StateGraph
from src.prompt_enhancer.graph.enhancer_node import prompt_enhancer_node
from src.prompt_enhancer.graph.state import PromptEnhancerState
def build_graph():
"""Build and return the prompt enhancer workflow graph."""
# Build state graph
builder = StateGraph(PromptEnhancerState)
# Add the enhancer node
builder.add_node("enhancer", prompt_enhancer_node)
# Set entry point
builder.set_entry_point("enhancer")
# Set finish point
builder.set_finish_point("enhancer")
# Compile and return the graph
return builder.compile()
@@ -0,0 +1,83 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import logging
import re
from langchain.schema import HumanMessage
from src.config.agents import AGENT_LLM_MAP
from src.llms.llm import get_llm_by_type
from src.prompts.template import apply_prompt_template
from src.prompt_enhancer.graph.state import PromptEnhancerState
logger = logging.getLogger(__name__)
def prompt_enhancer_node(state: PromptEnhancerState):
"""Node that enhances user prompts using AI analysis."""
logger.info("Enhancing user prompt...")
model = get_llm_by_type(AGENT_LLM_MAP["prompt_enhancer"])
try:
# Create messages with context if provided
context_info = ""
if state.get("context"):
context_info = f"\n\nAdditional context: {state['context']}"
original_prompt_message = HumanMessage(
content=f"Please enhance this prompt:{context_info}\n\nOriginal prompt: {state['prompt']}"
)
messages = apply_prompt_template(
"prompt_enhancer/prompt_enhancer",
{
"messages": [original_prompt_message],
"report_style": state.get("report_style"),
},
)
# Get the response from the model
response = model.invoke(messages)
# Extract content from response
response_content = response.content.strip()
logger.debug(f"Response content: {response_content}")
# Try to extract content from XML tags first
xml_match = re.search(
r"<enhanced_prompt>(.*?)</enhanced_prompt>", response_content, re.DOTALL
)
if xml_match:
# Extract content from XML tags and clean it up
enhanced_prompt = xml_match.group(1).strip()
logger.debug("Successfully extracted enhanced prompt from XML tags")
else:
# Fallback to original logic if no XML tags found
enhanced_prompt = response_content
logger.warning("No XML tags found in response, using fallback parsing")
# Remove common prefixes that might be added by the model
prefixes_to_remove = [
"Enhanced Prompt:",
"Enhanced prompt:",
"Here's the enhanced prompt:",
"Here is the enhanced prompt:",
"**Enhanced Prompt**:",
"**Enhanced prompt**:",
]
for prefix in prefixes_to_remove:
if enhanced_prompt.startswith(prefix):
enhanced_prompt = enhanced_prompt[len(prefix) :].strip()
break
logger.info("Prompt enhancement completed successfully")
logger.debug(f"Enhanced prompt: {enhanced_prompt}")
return {"output": enhanced_prompt}
except Exception as e:
logger.error(f"Error in prompt enhancement: {str(e)}")
return {"output": state["prompt"]}
+14
View File
@@ -0,0 +1,14 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from typing import TypedDict, Optional
from src.config.report_style import ReportStyle
class PromptEnhancerState(TypedDict):
"""State for the prompt enhancer workflow."""
prompt: str # Original prompt to enhance
context: Optional[str] # Additional context
report_style: Optional[ReportStyle] # Report style preference
output: Optional[str] # Enhanced prompt result
+24 -23
View File
@@ -57,14 +57,15 @@ Before creating a detailed plan, assess if there is sufficient context to answer
Different types of steps have different web search requirements:
1. **Research Steps** (`need_web_search: true`):
1. **Research Steps** (`need_search: true`):
- Retrieve information from the file with the URL with `rag://` or `http://` prefix specified by the user
- Gathering market data or industry trends
- Finding historical information
- Collecting competitor analysis
- Researching current events or news
- Finding statistical data or reports
2. **Data Processing Steps** (`need_web_search: false`):
2. **Data Processing Steps** (`need_search: false`):
- API calls and data extraction
- Database queries
- Raw data collection from existing sources
@@ -74,10 +75,10 @@ Different types of steps have different web search requirements:
## Exclusions
- **No Direct Calculations in Research Steps**:
- Research steps should only gather data and information
- All mathematical calculations must be handled by processing steps
- Numerical analysis must be delegated to processing steps
- Research steps focus on information gathering only
- Research steps should only gather data and information
- All mathematical calculations must be handled by processing steps
- Numerical analysis must be delegated to processing steps
- Research steps focus on information gathering only
## Analysis Framework
@@ -135,16 +136,16 @@ When planning information gathering, consider these key aspects and ensure COMPR
- To begin with, repeat user's requirement in your own words as `thought`.
- Rigorously assess if there is sufficient context to answer the question using the strict criteria above.
- If context is sufficient:
- Set `has_enough_context` to true
- No need to create information gathering steps
- Set `has_enough_context` to true
- No need to create information gathering steps
- If context is insufficient (default assumption):
- Break down the required information using the Analysis Framework
- Create NO MORE THAN {{ max_step_num }} focused and comprehensive steps that cover the most essential aspects
- Ensure each step is substantial and covers related information categories
- Prioritize breadth and depth within the {{ max_step_num }}-step constraint
- For each step, carefully assess if web search is needed:
- Research and external data gathering: Set `need_web_search: true`
- Internal data processing: Set `need_web_search: false`
- Break down the required information using the Analysis Framework
- Create NO MORE THAN {{ max_step_num }} focused and comprehensive steps that cover the most essential aspects
- Ensure each step is substantial and covers related information categories
- Prioritize breadth and depth within the {{ max_step_num }}-step constraint
- For each step, carefully assess if web search is needed:
- Research and external data gathering: Set `need_search: true`
- Internal data processing: Set `need_search: false`
- Specify the exact data to be collected in step's `description`. Include a `note` if necessary.
- Prioritize depth and volume of relevant information - limited information is not acceptable.
- Use the same language as the user to generate the plan.
@@ -156,10 +157,10 @@ Directly output the raw JSON format of `Plan` without "```json". The `Plan` inte
```ts
interface Step {
need_web_search: boolean; // Must be explicitly set for each step
need_search: boolean; // Must be explicitly set for each step
title: string;
description: string; // Specify exactly what data to collect
step_type: "research" | "processing"; // Indicates the nature of the step
description: string; // Specify exactly what data to collect. If the user input contains a link, please retain the full Markdown format when necessary.
step_type: "research" | "processing"; // Indicates the nature of the step
}
interface Plan {
@@ -167,7 +168,7 @@ interface Plan {
has_enough_context: boolean;
thought: string;
title: string;
steps: Step[]; // Research & Processing steps to get more context
steps: Step[]; // Research & Processing steps to get more context
}
```
@@ -179,8 +180,8 @@ interface Plan {
- Prioritize BOTH breadth (covering essential aspects) AND depth (detailed information on each aspect)
- Never settle for minimal information - the goal is a comprehensive, detailed final report
- Limited or insufficient information will lead to an inadequate final report
- Carefully assess each step's web search requirement based on its nature:
- Research steps (`need_web_search: true`) for gathering information
- Processing steps (`need_web_search: false`) for calculations and data processing
- Carefully assess each step's web search or retrieve from URL requirement based on its nature:
- Research steps (`need_search: true`) for gathering information
- Processing steps (`need_search: false`) for calculations and data processing
- Default to gathering more information unless the strictest sufficient context criteria are met
- Always use the language specified by the locale = **{{ locale }}**.
- Always use the language specified by the locale = **{{ locale }}**.
+2 -4
View File
@@ -13,9 +13,7 @@ class StepType(str, Enum):
class Step(BaseModel):
need_web_search: bool = Field(
..., description="Must be explicitly set for each step"
)
need_search: bool = Field(..., description="Must be explicitly set for each step")
title: str
description: str = Field(..., description="Specify exactly what data to collect")
step_type: StepType = Field(..., description="Indicates the nature of the step")
@@ -47,7 +45,7 @@ class Plan(BaseModel):
"title": "AI Market Research Plan",
"steps": [
{
"need_web_search": True,
"need_search": True,
"title": "Current AI Market Analysis",
"description": (
"Collect data on market size, growth rates, major players, and investment trends in AI sector."
@@ -0,0 +1,135 @@
---
CURRENT_TIME: {{ CURRENT_TIME }}
---
You are an expert prompt engineer. Your task is to enhance user prompts to make them more effective, specific, and likely to produce high-quality results from AI systems.
# Your Role
- Analyze the original prompt for clarity, specificity, and completeness
- Enhance the prompt by adding relevant details, context, and structure
- Make the prompt more actionable and results-oriented
- Preserve the user's original intent while improving effectiveness
{% if report_style == "academic" %}
# Enhancement Guidelines for Academic Style
1. **Add methodological rigor**: Include research methodology, scope, and analytical framework
2. **Specify academic structure**: Organize with clear thesis, literature review, analysis, and conclusions
3. **Clarify scholarly expectations**: Specify citation requirements, evidence standards, and academic tone
4. **Add theoretical context**: Include relevant theoretical frameworks and disciplinary perspectives
5. **Ensure precision**: Use precise terminology and avoid ambiguous language
6. **Include limitations**: Acknowledge scope limitations and potential biases
{% elif report_style == "popular_science" %}
# Enhancement Guidelines for Popular Science Style
1. **Add accessibility**: Transform technical concepts into relatable analogies and examples
2. **Improve narrative structure**: Organize as an engaging story with clear beginning, middle, and end
3. **Clarify audience expectations**: Specify general audience level and engagement goals
4. **Add human context**: Include real-world applications and human interest elements
5. **Make it compelling**: Ensure the prompt guides toward fascinating and wonder-inspiring content
6. **Include visual elements**: Suggest use of metaphors and descriptive language for complex concepts
{% elif report_style == "news" %}
# Enhancement Guidelines for News Style
1. **Add journalistic rigor**: Include fact-checking requirements, source verification, and objectivity standards
2. **Improve news structure**: Organize with inverted pyramid structure (most important information first)
3. **Clarify reporting expectations**: Specify timeliness, accuracy, and balanced perspective requirements
4. **Add contextual background**: Include relevant background information and broader implications
5. **Make it newsworthy**: Ensure the prompt focuses on current relevance and public interest
6. **Include attribution**: Specify source requirements and quote standards
{% elif report_style == "social_media" %}
# Enhancement Guidelines for Social Media Style
1. **Add engagement focus**: Include attention-grabbing elements, hooks, and shareability factors
2. **Improve platform structure**: Organize for specific platform requirements (character limits, hashtags, etc.)
3. **Clarify audience expectations**: Specify target demographic and engagement goals
4. **Add viral elements**: Include trending topics, relatable content, and interactive elements
5. **Make it shareable**: Ensure the prompt guides toward content that encourages sharing and discussion
6. **Include visual considerations**: Suggest emoji usage, formatting, and visual appeal elements
{% else %}
# General Enhancement Guidelines
1. **Add specificity**: Include relevant details, scope, and constraints
2. **Improve structure**: Organize the request logically with clear sections if needed
3. **Clarify expectations**: Specify desired output format, length, or style
4. **Add context**: Include background information that would help generate better results
5. **Make it actionable**: Ensure the prompt guides toward concrete, useful outputs
{% endif %}
# Output Requirements
- You may include thoughts or reasoning before your final answer
- Wrap the final enhanced prompt in XML tags: <enhanced_prompt></enhanced_prompt>
- Do NOT include any explanations, comments, or meta-text within the XML tags
- Do NOT use phrases like "Enhanced Prompt:" or "Here's the enhanced version:" within the XML tags
- The content within the XML tags should be ready to use directly as a prompt
{% if report_style == "academic" %}
# Academic Style Examples
**Original**: "Write about AI"
**Enhanced**:
<enhanced_prompt>
Conduct a comprehensive academic analysis of artificial intelligence applications across three key sectors: healthcare, education, and business. Employ a systematic literature review methodology to examine peer-reviewed sources from the past five years. Structure your analysis with: (1) theoretical framework defining AI and its taxonomies, (2) sector-specific case studies with quantitative performance metrics, (3) critical evaluation of implementation challenges and ethical considerations, (4) comparative analysis across sectors, and (5) evidence-based recommendations for future research directions. Maintain academic rigor with proper citations, acknowledge methodological limitations, and present findings with appropriate hedging language. Target length: 3000-4000 words with APA formatting.
</enhanced_prompt>
**Original**: "Explain climate change"
**Enhanced**:
<enhanced_prompt>
Provide a rigorous academic examination of anthropogenic climate change, synthesizing current scientific consensus and recent research developments. Structure your analysis as follows: (1) theoretical foundations of greenhouse effect and radiative forcing mechanisms, (2) systematic review of empirical evidence from paleoclimatic, observational, and modeling studies, (3) critical analysis of attribution studies linking human activities to observed warming, (4) evaluation of climate sensitivity estimates and uncertainty ranges, (5) assessment of projected impacts under different emission scenarios, and (6) discussion of research gaps and methodological limitations. Include quantitative data, statistical significance levels, and confidence intervals where appropriate. Cite peer-reviewed sources extensively and maintain objective, third-person academic voice throughout.
</enhanced_prompt>
{% elif report_style == "popular_science" %}
# Popular Science Style Examples
**Original**: "Write about AI"
**Enhanced**:
<enhanced_prompt>
Tell the fascinating story of how artificial intelligence is quietly revolutionizing our daily lives in ways most people never realize. Take readers on an engaging journey through three surprising realms: the hospital where AI helps doctors spot diseases faster than ever before, the classroom where intelligent tutors adapt to each student's learning style, and the boardroom where algorithms are making million-dollar decisions. Use vivid analogies (like comparing neural networks to how our brains work) and real-world examples that readers can relate to. Include 'wow factor' moments that showcase AI's incredible capabilities, but also honest discussions about current limitations. Write with infectious enthusiasm while maintaining scientific accuracy, and conclude with exciting possibilities that await us in the near future. Aim for 1500-2000 words that feel like a captivating conversation with a brilliant friend.
</enhanced_prompt>
**Original**: "Explain climate change"
**Enhanced**:
<enhanced_prompt>
Craft a compelling narrative that transforms the complex science of climate change into an accessible and engaging story for curious readers. Begin with a relatable scenario (like why your hometown weather feels different than when you were a kid) and use this as a gateway to explore the fascinating science behind our changing planet. Employ vivid analogies - compare Earth's atmosphere to a blanket, greenhouse gases to invisible heat-trapping molecules, and climate feedback loops to a snowball rolling downhill. Include surprising facts and 'aha moments' that will make readers think differently about the world around them. Weave in human stories of scientists making discoveries, communities adapting to change, and innovative solutions being developed. Balance the serious implications with hope and actionable insights, concluding with empowering steps readers can take. Write with wonder and curiosity, making complex concepts feel approachable and personally relevant.
</enhanced_prompt>
{% elif report_style == "news" %}
# News Style Examples
**Original**: "Write about AI"
**Enhanced**:
<enhanced_prompt>
Report on the current state and immediate impact of artificial intelligence across three critical sectors: healthcare, education, and business. Lead with the most newsworthy developments and recent breakthroughs that are affecting people today. Structure using inverted pyramid format: start with key findings and immediate implications, then provide essential background context, followed by detailed analysis and expert perspectives. Include specific, verifiable data points, recent statistics, and quotes from credible sources including industry leaders, researchers, and affected stakeholders. Address both benefits and concerns with balanced reporting, fact-check all claims, and provide proper attribution for all information. Focus on timeliness and relevance to current events, highlighting what's happening now and what readers need to know. Maintain journalistic objectivity while making the significance clear to a general news audience. Target 800-1200 words following AP style guidelines.
</enhanced_prompt>
**Original**: "Explain climate change"
**Enhanced**:
<enhanced_prompt>
Provide comprehensive news coverage of climate change that explains the current scientific understanding and immediate implications for readers. Lead with the most recent and significant developments in climate science, policy, or impacts that are making headlines today. Structure the report with: breaking developments first, essential background for understanding the issue, current scientific consensus with specific data and timeframes, real-world impacts already being observed, policy responses and debates, and what experts say comes next. Include quotes from credible climate scientists, policy makers, and affected communities. Present information objectively while clearly communicating the scientific consensus, fact-check all claims, and provide proper source attribution. Address common misconceptions with factual corrections. Focus on what's happening now, why it matters to readers, and what they can expect in the near future. Follow journalistic standards for accuracy, balance, and timeliness.
</enhanced_prompt>
{% elif report_style == "social_media" %}
# Social Media Style Examples
**Original**: "Write about AI"
**Enhanced**:
<enhanced_prompt>
Create engaging social media content about AI that will stop the scroll and spark conversations! Start with an attention-grabbing hook like 'You won't believe what AI just did in hospitals this week 🤯' and structure as a compelling thread or post series. Include surprising facts, relatable examples (like AI helping doctors spot diseases or personalizing your Netflix recommendations), and interactive elements that encourage sharing and comments. Use strategic hashtags (#AI #Technology #Future), incorporate relevant emojis for visual appeal, and include questions that prompt audience engagement ('Have you noticed AI in your daily life? Drop examples below! 👇'). Make complex concepts digestible with bite-sized explanations, trending analogies, and shareable quotes. Include a clear call-to-action and optimize for the specific platform (Twitter threads, Instagram carousel, LinkedIn professional insights, or TikTok-style quick facts). Aim for high shareability with content that feels both informative and entertaining.
</enhanced_prompt>
**Original**: "Explain climate change"
**Enhanced**:
<enhanced_prompt>
Develop viral-worthy social media content that makes climate change accessible and shareable without being preachy. Open with a scroll-stopping hook like 'The weather app on your phone is telling a bigger story than you think 📱🌡️' and break down complex science into digestible, engaging chunks. Use relatable comparisons (Earth's fever, atmosphere as a blanket), trending formats (before/after visuals, myth-busting series, quick facts), and interactive elements (polls, questions, challenges). Include strategic hashtags (#ClimateChange #Science #Environment), eye-catching emojis, and shareable graphics or infographics. Address common questions and misconceptions with clear, factual responses. Create content that encourages positive action rather than climate anxiety, ending with empowering steps followers can take. Optimize for platform-specific features (Instagram Stories, TikTok trends, Twitter threads) and include calls-to-action that drive engagement and sharing.
</enhanced_prompt>
{% else %}
# General Examples
**Original**: "Write about AI"
**Enhanced**:
<enhanced_prompt>
Write a comprehensive 1000-word analysis of artificial intelligence's current applications in healthcare, education, and business. Include specific examples of AI tools being used in each sector, discuss both benefits and challenges, and provide insights into future trends. Structure the response with clear sections for each industry and conclude with key takeaways.
</enhanced_prompt>
**Original**: "Explain climate change"
**Enhanced**:
<enhanced_prompt>
Provide a detailed explanation of climate change suitable for a general audience. Cover the scientific mechanisms behind global warming, major causes including greenhouse gas emissions, observable effects we're seeing today, and projected future impacts. Include specific data and examples, and explain the difference between weather and climate. Organize the response with clear headings and conclude with actionable steps individuals can take.
</enhanced_prompt>
{% endif %}
+159 -2
View File
@@ -2,7 +2,21 @@
CURRENT_TIME: {{ CURRENT_TIME }}
---
You are a professional reporter responsible for writing clear, comprehensive reports based ONLY on provided information and verifiable facts.
{% if report_style == "academic" %}
You are a distinguished academic researcher and scholarly writer. Your report must embody the highest standards of academic rigor and intellectual discourse. Write with the precision of a peer-reviewed journal article, employing sophisticated analytical frameworks, comprehensive literature synthesis, and methodological transparency. Your language should be formal, technical, and authoritative, utilizing discipline-specific terminology with exactitude. Structure arguments logically with clear thesis statements, supporting evidence, and nuanced conclusions. Maintain complete objectivity, acknowledge limitations, and present balanced perspectives on controversial topics. The report should demonstrate deep scholarly engagement and contribute meaningfully to academic knowledge.
{% elif report_style == "popular_science" %}
You are an award-winning science communicator and storyteller. Your mission is to transform complex scientific concepts into captivating narratives that spark curiosity and wonder in everyday readers. Write with the enthusiasm of a passionate educator, using vivid analogies, relatable examples, and compelling storytelling techniques. Your tone should be warm, approachable, and infectious in its excitement about discovery. Break down technical jargon into accessible language without sacrificing accuracy. Use metaphors, real-world comparisons, and human interest angles to make abstract concepts tangible. Think like a National Geographic writer or a TED Talk presenter - engaging, enlightening, and inspiring.
{% elif report_style == "news" %}
You are an NBC News correspondent and investigative journalist with decades of experience in breaking news and in-depth reporting. Your report must exemplify the gold standard of American broadcast journalism: authoritative, meticulously researched, and delivered with the gravitas and credibility that NBC News is known for. Write with the precision of a network news anchor, employing the classic inverted pyramid structure while weaving compelling human narratives. Your language should be clear, authoritative, and accessible to prime-time television audiences. Maintain NBC's tradition of balanced reporting, thorough fact-checking, and ethical journalism. Think like Lester Holt or Andrea Mitchell - delivering complex stories with clarity, context, and unwavering integrity.
{% elif report_style == "social_media" %}
{% if locale == "zh-CN" %}
You are a popular 小红书 (Xiaohongshu) content creator specializing in lifestyle and knowledge sharing. Your report should embody the authentic, personal, and engaging style that resonates with 小红书 users. Write with genuine enthusiasm and a "姐妹们" (sisters) tone, as if sharing exciting discoveries with close friends. Use abundant emojis, create "种草" (grass-planting/recommendation) moments, and structure content for easy mobile consumption. Your writing should feel like a personal diary entry mixed with expert insights - warm, relatable, and irresistibly shareable. Think like a top 小红书 blogger who effortlessly combines personal experience with valuable information, making readers feel like they've discovered a hidden gem.
{% else %}
You are a viral Twitter content creator and digital influencer specializing in breaking down complex topics into engaging, shareable threads. Your report should be optimized for maximum engagement and viral potential across social media platforms. Write with energy, authenticity, and a conversational tone that resonates with global online communities. Use strategic hashtags, create quotable moments, and structure content for easy consumption and sharing. Think like a successful Twitter thought leader who can make any topic accessible, engaging, and discussion-worthy while maintaining credibility and accuracy.
{% endif %}
{% else %}
You are a professional reporter responsible for writing clear, comprehensive reports based ONLY on provided information and verifiable facts. Your report should adopt a professional tone.
{% endif %}
# Role
@@ -43,10 +57,40 @@ Structure your report in the following format:
- **Including images from the previous steps in the report is very helpful.**
5. **Survey Note** (for more comprehensive reports)
{% if report_style == "academic" %}
- **Literature Review & Theoretical Framework**: Comprehensive analysis of existing research and theoretical foundations
- **Methodology & Data Analysis**: Detailed examination of research methods and analytical approaches
- **Critical Discussion**: In-depth evaluation of findings with consideration of limitations and implications
- **Future Research Directions**: Identification of gaps and recommendations for further investigation
{% elif report_style == "popular_science" %}
- **The Bigger Picture**: How this research fits into the broader scientific landscape
- **Real-World Applications**: Practical implications and potential future developments
- **Behind the Scenes**: Interesting details about the research process and challenges faced
- **What's Next**: Exciting possibilities and upcoming developments in the field
{% elif report_style == "news" %}
- **NBC News Analysis**: In-depth examination of the story's broader implications and significance
- **Impact Assessment**: How these developments affect different communities, industries, and stakeholders
- **Expert Perspectives**: Insights from credible sources, analysts, and subject matter experts
- **Timeline & Context**: Chronological background and historical context essential for understanding
- **What's Next**: Expected developments, upcoming milestones, and stories to watch
{% elif report_style == "social_media" %}
{% if locale == "zh-CN" %}
- **【种草时刻】**: 最值得关注的亮点和必须了解的核心信息
- **【数据震撼】**: 用小红书风格展示重要统计数据和发现
- **【姐妹们的看法】**: 社区热议话题和大家的真实反馈
- **【行动指南】**: 实用建议和读者可以立即行动的清单
{% else %}
- **Thread Highlights**: Key takeaways formatted for maximum shareability
- **Data That Matters**: Important statistics and findings presented for viral potential
- **Community Pulse**: Trending discussions and reactions from the online community
- **Action Steps**: Practical advice and immediate next steps for readers
{% endif %}
{% else %}
- A more detailed, academic-style analysis.
- Include comprehensive sections covering all aspects of the topic.
- Can include comparative analysis, tables, and detailed feature breakdowns.
- This section is optional for shorter reports.
{% endif %}
6. **Key Citations**
- List all references at the end in link reference format.
@@ -56,7 +100,64 @@ Structure your report in the following format:
# Writing Guidelines
1. Writing style:
- Use professional tone.
{% if report_style == "academic" %}
**Academic Excellence Standards:**
- Employ sophisticated, formal academic discourse with discipline-specific terminology
- Construct complex, nuanced arguments with clear thesis statements and logical progression
- Use third-person perspective and passive voice where appropriate for objectivity
- Include methodological considerations and acknowledge research limitations
- Reference theoretical frameworks and cite relevant scholarly work patterns
- Maintain intellectual rigor with precise, unambiguous language
- Avoid contractions, colloquialisms, and informal expressions entirely
- Use hedging language appropriately ("suggests," "indicates," "appears to")
{% elif report_style == "popular_science" %}
**Science Communication Excellence:**
- Write with infectious enthusiasm and genuine curiosity about discoveries
- Transform technical jargon into vivid, relatable analogies and metaphors
- Use active voice and engaging narrative techniques to tell scientific stories
- Include "wow factor" moments and surprising revelations to maintain interest
- Employ conversational tone while maintaining scientific accuracy
- Use rhetorical questions to engage readers and guide their thinking
- Include human elements: researcher personalities, discovery stories, real-world impacts
- Balance accessibility with intellectual respect for your audience
{% elif report_style == "news" %}
**NBC News Editorial Standards:**
- Open with a compelling lede that captures the essence of the story in 25-35 words
- Use the classic inverted pyramid: most newsworthy information first, supporting details follow
- Write in clear, conversational broadcast style that sounds natural when read aloud
- Employ active voice and strong, precise verbs that convey action and urgency
- Attribute every claim to specific, credible sources using NBC's attribution standards
- Use present tense for ongoing situations, past tense for completed events
- Maintain NBC's commitment to balanced reporting with multiple perspectives
- Include essential context and background without overwhelming the main story
- Verify information through at least two independent sources when possible
- Clearly label speculation, analysis, and ongoing investigations
- Use transitional phrases that guide readers smoothly through the narrative
{% elif report_style == "social_media" %}
{% if locale == "zh-CN" %}
**小红书风格写作标准:**
- 用"姐妹们!"、"宝子们!"等亲切称呼开头,营造闺蜜聊天氛围
- 大量使用emoji表情符号增强表达力和视觉吸引力 ✨
- 采用"种草"语言:"真的绝了!"、"必须安利给大家!"、"不看后悔系列!"
- 使用小红书特色标题格式:"【干货分享】"、"【亲测有效】"、"【避雷指南】"
- 穿插个人感受和体验:"我当时看到这个数据真的震惊了!"
- 用数字和符号增强视觉效果:①②③、✅❌、🔥💡⭐
- 创造"金句"和可截图分享的内容段落
- 结尾用互动性语言:"你们觉得呢?"、"评论区聊聊!"、"记得点赞收藏哦!"
{% else %}
**Twitter/X Engagement Standards:**
- Open with attention-grabbing hooks that stop the scroll
- Use thread-style formatting with numbered points (1/n, 2/n, etc.)
- Incorporate strategic hashtags for discoverability and trending topics
- Write quotable, tweetable snippets that beg to be shared
- Use conversational, authentic voice with personality and wit
- Include relevant emojis to enhance meaning and visual appeal 🧵📊💡
- Create "thread-worthy" content with clear progression and payoff
- End with engagement prompts: "What do you think?", "Retweet if you agree"
{% endif %}
{% else %}
- Use a professional tone.
{% endif %}
- Be concise and precise.
- Avoid speculation.
- Support claims with evidence.
@@ -77,6 +178,62 @@ Structure your report in the following format:
- Use horizontal rules (---) to separate major sections.
- Track the sources of information but keep the main text clean and readable.
{% if report_style == "academic" %}
**Academic Formatting Specifications:**
- Use formal section headings with clear hierarchical structure (## Introduction, ### Methodology, #### Subsection)
- Employ numbered lists for methodological steps and logical sequences
- Use block quotes for important definitions or key theoretical concepts
- Include detailed tables with comprehensive headers and statistical data
- Use footnote-style formatting for additional context or clarifications
- Maintain consistent academic citation patterns throughout
- Use `code blocks` for technical specifications, formulas, or data samples
{% elif report_style == "popular_science" %}
**Science Communication Formatting:**
- Use engaging, descriptive headings that spark curiosity ("The Surprising Discovery That Changed Everything")
- Employ creative formatting like callout boxes for "Did You Know?" facts
- Use bullet points for easy-to-digest key findings
- Include visual breaks with strategic use of bold text for emphasis
- Format analogies and metaphors prominently to aid understanding
- Use numbered lists for step-by-step explanations of complex processes
- Highlight surprising statistics or findings with special formatting
{% elif report_style == "news" %}
**NBC News Formatting Standards:**
- Craft headlines that are informative yet compelling, following NBC's style guide
- Use NBC-style datelines and bylines for professional credibility
- Structure paragraphs for broadcast readability (1-2 sentences for digital, 2-3 for print)
- Employ strategic subheadings that advance the story narrative
- Format direct quotes with proper attribution and context
- Use bullet points sparingly, primarily for breaking news updates or key facts
- Include "BREAKING" or "DEVELOPING" labels for ongoing stories
- Format source attribution clearly: "according to NBC News," "sources tell NBC News"
- Use italics for emphasis on key terms or breaking developments
- Structure the story with clear sections: Lede, Context, Analysis, Looking Ahead
{% elif report_style == "social_media" %}
{% if locale == "zh-CN" %}
**小红书格式优化标准:**
- 使用吸睛标题配合emoji:"🔥【重磅】这个发现太震撼了!"
- 关键数据用醒目格式突出:「 重点数据 」或 ⭐ 核心发现 ⭐
- 适度使用大写强调:真的YYDS!、绝绝子!
- 用emoji作为分点符号:✨、🌟、、💯
- 创建话题标签区域:#科技前沿 #必看干货 #涨知识了
- 设置"划重点"总结区域,方便快速阅读
- 利用换行和空白营造手机阅读友好的版式
- 制作"金句卡片"格式,便于截图分享
- 使用分割线和特殊符号:「」『』【】━━━━━━
{% else %}
**Twitter/X Formatting Standards:**
- Use compelling headlines with strategic emoji placement 🧵⚡️🔥
- Format key insights as standalone, quotable tweet blocks
- Employ thread numbering for multi-part content (1/12, 2/12, etc.)
- Use bullet points with emoji bullets for visual appeal
- Include strategic hashtags at the end: #TechNews #Innovation #MustRead
- Create "TL;DR" summaries for quick consumption
- Use line breaks and white space for mobile readability
- Format "quotable moments" with clear visual separation
- Include call-to-action elements: "🔄 RT to share" "💬 What's your take?"
{% endif %}
{% endif %}
# Data Integrity
- Only use information explicitly provided in the input.
+4 -1
View File
@@ -11,6 +11,9 @@ You are dedicated to conducting thorough investigations using search tools and p
You have access to two types of tools:
1. **Built-in Tools**: These are always available:
{% if resources %}
- **local_search_tool**: For retrieving information from the local knowledge base when user mentioned in the messages.
{% endif %}
- **web_search_tool**: For performing web searches
- **crawl_tool**: For reading content from URLs
@@ -34,7 +37,7 @@ You have access to two types of tools:
3. **Plan the Solution**: Determine the best approach to solve the problem using the available tools.
4. **Execute the Solution**:
- Forget your previous knowledge, so you **should leverage the tools** to retrieve the information.
- Use the **web_search_tool** or other suitable search tool to perform a search with the provided keywords.
- Use the {% if resources %}**local_search_tool** or{% endif %}**web_search_tool** or other suitable search tool to perform a search with the provided keywords.
- When the task includes time range requirements:
- Incorporate appropriate time-based search parameters in your queries (e.g., "after:2020", "before:2023", or specific date ranges)
- Ensure search results respect the specified time constraints.
+17
View File
@@ -0,0 +1,17 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from .retriever import Retriever, Document, Resource, Chunk
from .ragflow import RAGFlowProvider
from .vikingdb_knowledge_base import VikingDBKnowledgeBaseProvider
from .builder import build_retriever
__all__ = [
Retriever,
Document,
Resource,
RAGFlowProvider,
VikingDBKnowledgeBaseProvider,
Chunk,
build_retriever,
]
+17
View File
@@ -0,0 +1,17 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from src.config.tools import SELECTED_RAG_PROVIDER, RAGProvider
from src.rag.ragflow import RAGFlowProvider
from src.rag.vikingdb_knowledge_base import VikingDBKnowledgeBaseProvider
from src.rag.retriever import Retriever
def build_retriever() -> Retriever | None:
if SELECTED_RAG_PROVIDER == RAGProvider.RAGFLOW.value:
return RAGFlowProvider()
elif SELECTED_RAG_PROVIDER == RAGProvider.VIKINGDB_KNOWLEDGE_BASE.value:
return VikingDBKnowledgeBaseProvider()
elif SELECTED_RAG_PROVIDER:
raise ValueError(f"Unsupported RAG provider: {SELECTED_RAG_PROVIDER}")
return None
+124
View File
@@ -0,0 +1,124 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import os
import requests
from src.rag.retriever import Chunk, Document, Resource, Retriever
from urllib.parse import urlparse
class RAGFlowProvider(Retriever):
"""
RAGFlowProvider is a provider that uses RAGFlow to retrieve documents.
"""
api_url: str
api_key: str
page_size: int = 10
def __init__(self):
api_url = os.getenv("RAGFLOW_API_URL")
if not api_url:
raise ValueError("RAGFLOW_API_URL is not set")
self.api_url = api_url
api_key = os.getenv("RAGFLOW_API_KEY")
if not api_key:
raise ValueError("RAGFLOW_API_KEY is not set")
self.api_key = api_key
page_size = os.getenv("RAGFLOW_PAGE_SIZE")
if page_size:
self.page_size = int(page_size)
def query_relevant_documents(
self, query: str, resources: list[Resource] = []
) -> list[Document]:
headers = {
"Authorization": f"Bearer {self.api_key}",
"Content-Type": "application/json",
}
dataset_ids: list[str] = []
document_ids: list[str] = []
for resource in resources:
dataset_id, document_id = parse_uri(resource.uri)
dataset_ids.append(dataset_id)
if document_id:
document_ids.append(document_id)
payload = {
"question": query,
"dataset_ids": dataset_ids,
"document_ids": document_ids,
"page_size": self.page_size,
}
response = requests.post(
f"{self.api_url}/api/v1/retrieval", headers=headers, json=payload
)
if response.status_code != 200:
raise Exception(f"Failed to query documents: {response.text}")
result = response.json()
data = result.get("data", {})
doc_aggs = data.get("doc_aggs", [])
docs: dict[str, Document] = {
doc.get("doc_id"): Document(
id=doc.get("doc_id"),
title=doc.get("doc_name"),
chunks=[],
)
for doc in doc_aggs
}
for chunk in data.get("chunks", []):
doc = docs.get(chunk.get("document_id"))
if doc:
doc.chunks.append(
Chunk(
content=chunk.get("content"),
similarity=chunk.get("similarity"),
)
)
return list(docs.values())
def list_resources(self, query: str | None = None) -> list[Resource]:
headers = {
"Authorization": f"Bearer {self.api_key}",
"Content-Type": "application/json",
}
params = {}
if query:
params["name"] = query
response = requests.get(
f"{self.api_url}/api/v1/datasets", headers=headers, params=params
)
if response.status_code != 200:
raise Exception(f"Failed to list resources: {response.text}")
result = response.json()
resources = []
for item in result.get("data", []):
item = Resource(
uri=f"rag://dataset/{item.get('id')}",
title=item.get("name", ""),
description=item.get("description", ""),
)
resources.append(item)
return resources
def parse_uri(uri: str) -> tuple[str, str]:
parsed = urlparse(uri)
if parsed.scheme != "rag":
raise ValueError(f"Invalid URI: {uri}")
return parsed.path.split("/")[1], parsed.fragment
+80
View File
@@ -0,0 +1,80 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import abc
from pydantic import BaseModel, Field
class Chunk:
content: str
similarity: float
def __init__(self, content: str, similarity: float):
self.content = content
self.similarity = similarity
class Document:
"""
Document is a class that represents a document.
"""
id: str
url: str | None = None
title: str | None = None
chunks: list[Chunk] = []
def __init__(
self,
id: str,
url: str | None = None,
title: str | None = None,
chunks: list[Chunk] = [],
):
self.id = id
self.url = url
self.title = title
self.chunks = chunks
def to_dict(self) -> dict:
d = {
"id": self.id,
"content": "\n\n".join([chunk.content for chunk in self.chunks]),
}
if self.url:
d["url"] = self.url
if self.title:
d["title"] = self.title
return d
class Resource(BaseModel):
"""
Resource is a class that represents a resource.
"""
uri: str = Field(..., description="The URI of the resource")
title: str = Field(..., description="The title of the resource")
description: str | None = Field("", description="The description of the resource")
class Retriever(abc.ABC):
"""
Define a RAG provider, which can be used to query documents and resources.
"""
@abc.abstractmethod
def list_resources(self, query: str | None = None) -> list[Resource]:
"""
List resources from the rag provider.
"""
pass
@abc.abstractmethod
def query_relevant_documents(
self, query: str, resources: list[Resource] = []
) -> list[Document]:
"""
Query relevant documents from the resources.
"""
pass
+208
View File
@@ -0,0 +1,208 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import os
import requests
import json
from src.rag.retriever import Chunk, Document, Resource, Retriever
from urllib.parse import urlparse
from volcengine.auth.SignerV4 import SignerV4
from volcengine.base.Request import Request
from volcengine.Credentials import Credentials
class VikingDBKnowledgeBaseProvider(Retriever):
"""
VikingDBKnowledgeBaseProvider is a provider that uses VikingDB Knowledge base API to retrieve documents.
"""
api_url: str
api_ak: str
api_sk: str
retrieval_size: int = 10
def __init__(self):
api_url = os.getenv("VIKINGDB_KNOWLEDGE_BASE_API_URL")
if not api_url:
raise ValueError("VIKINGDB_KNOWLEDGE_BASE_API_URL is not set")
self.api_url = api_url
api_ak = os.getenv("VIKINGDB_KNOWLEDGE_BASE_API_AK")
if not api_ak:
raise ValueError("VIKINGDB_KNOWLEDGE_BASE_API_AK is not set")
self.api_ak = api_ak
api_sk = os.getenv("VIKINGDB_KNOWLEDGE_BASE_API_SK")
if not api_sk:
raise ValueError("VIKINGDB_KNOWLEDGE_BASE_API_SK is not set")
self.api_sk = api_sk
retrieval_size = os.getenv("VIKINGDB_KNOWLEDGE_BASE_RETRIEVAL_SIZE")
if retrieval_size:
self.retrieval_size = int(retrieval_size)
def prepare_request(self, method, path, params=None, data=None, doseq=0):
"""
Prepare signed request using volcengine auth
"""
if params:
for key in params:
if (
type(params[key]) is int
or type(params[key]) is float
or type(params[key]) is bool
):
params[key] = str(params[key])
elif type(params[key]) is list:
if not doseq:
params[key] = ",".join(params[key])
r = Request()
r.set_shema("https")
r.set_method(method)
r.set_connection_timeout(10)
r.set_socket_timeout(10)
mheaders = {
"Accept": "application/json",
"Content-Type": "application/json",
}
r.set_headers(mheaders)
if params:
r.set_query(params)
r.set_path(path)
if data is not None:
r.set_body(json.dumps(data))
credentials = Credentials(self.api_ak, self.api_sk, "air", "cn-north-1")
SignerV4.sign(r, credentials)
return r
def query_relevant_documents(
self, query: str, resources: list[Resource] = []
) -> list[Document]:
"""
Query relevant documents from the knowledge base
"""
if not resources:
return []
all_documents = {}
for resource in resources:
resource_id, document_id = parse_uri(resource.uri)
request_params = {
"resource_id": resource_id,
"query": query,
"limit": self.retrieval_size,
"dense_weight": 0.5,
"pre_processing": {
"need_instruction": True,
"rewrite": False,
"return_token_usage": True,
},
"post_processing": {
"rerank_switch": True,
"chunk_diffusion_count": 0,
"chunk_group": True,
"get_attachment_link": True,
},
}
if document_id:
doc_filter = {"op": "must", "field": "doc_id", "conds": [document_id]}
query_param = {"doc_filter": doc_filter}
request_params["query_param"] = query_param
method = "POST"
path = "/api/knowledge/collection/search_knowledge"
info_req = self.prepare_request(
method=method, path=path, data=request_params
)
rsp = requests.request(
method=info_req.method,
url="http://{}{}".format(self.api_url, info_req.path),
headers=info_req.headers,
data=info_req.body,
)
try:
response = json.loads(rsp.text)
except json.JSONDecodeError as e:
raise ValueError(f"Failed to parse JSON response: {e}")
if response["code"] != 0:
raise ValueError(
f"Failed to query documents from resource: {response['message']}"
)
rsp_data = response.get("data", {})
if "result_list" not in rsp_data:
continue
result_list = rsp_data["result_list"]
for item in result_list:
doc_info = item.get("doc_info", {})
doc_id = doc_info.get("doc_id")
if not doc_id:
continue
if doc_id not in all_documents:
all_documents[doc_id] = Document(
id=doc_id, title=doc_info.get("doc_name"), chunks=[]
)
chunk = Chunk(
content=item.get("content", ""), similarity=item.get("score", 0.0)
)
all_documents[doc_id].chunks.append(chunk)
return list(all_documents.values())
def list_resources(self, query: str | None = None) -> list[Resource]:
"""
List resources (knowledge bases) from the knowledge base service
"""
method = "POST"
path = "/api/knowledge/collection/list"
info_req = self.prepare_request(method=method, path=path)
rsp = requests.request(
method=info_req.method,
url="http://{}{}".format(self.api_url, info_req.path),
headers=info_req.headers,
data=info_req.body,
)
try:
response = json.loads(rsp.text)
except json.JSONDecodeError as e:
raise ValueError(f"Failed to parse JSON response: {e}")
if response["code"] != 0:
raise Exception(f"Failed to list resources: {response["message"]}")
resources = []
rsp_data = response.get("data", {})
collection_list = rsp_data.get("collection_list", [])
for item in collection_list:
collection_name = item.get("collection_name", "")
description = item.get("description", "")
if query and query.lower() not in collection_name.lower():
continue
resource_id = item.get("resource_id", "")
resource = Resource(
uri=f"rag://dataset/{resource_id}",
title=collection_name,
description=description,
)
resources.append(resource)
return resources
def parse_uri(uri: str) -> tuple[str, str]:
parsed = urlparse(uri)
if parsed.scheme != "rag":
raise ValueError(f"Invalid URI: {uri}")
return parsed.path.split("/")[1], parsed.fragment
+162 -32
View File
@@ -5,33 +5,47 @@ import base64
import json
import logging
import os
from typing import List, cast
from typing import Annotated, List, cast
from uuid import uuid4
from fastapi import FastAPI, HTTPException
from fastapi import FastAPI, HTTPException, Query
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import Response, StreamingResponse
from langchain_core.messages import AIMessageChunk, ToolMessage
from langchain_core.messages import AIMessageChunk, BaseMessage, ToolMessage
from langgraph.types import Command
from src.config.report_style import ReportStyle
from src.config.tools import SELECTED_RAG_PROVIDER
from src.graph.builder import build_graph_with_memory
from src.llms.llm import get_configured_llm_models
from src.podcast.graph.builder import build_graph as build_podcast_graph
from src.ppt.graph.builder import build_graph as build_ppt_graph
from src.prompt_enhancer.graph.builder import build_graph as build_prompt_enhancer_graph
from src.prose.graph.builder import build_graph as build_prose_graph
from src.rag.builder import build_retriever
from src.rag.retriever import Resource
from src.server.chat_request import (
ChatMessage,
ChatRequest,
EnhancePromptRequest,
GeneratePodcastRequest,
GeneratePPTRequest,
GenerateProseRequest,
TTSRequest,
)
from src.server.config_request import ConfigResponse
from src.server.mcp_request import MCPServerMetadataRequest, MCPServerMetadataResponse
from src.server.mcp_utils import load_mcp_tools
from src.server.rag_request import (
RAGConfigResponse,
RAGResourceRequest,
RAGResourcesResponse,
)
from src.tools import VolcengineTTS
logger = logging.getLogger(__name__)
INTERNAL_SERVER_ERROR_DETAIL = "Internal Server Error"
app = FastAPI(
title="DeerFlow API",
description="API for Deer",
@@ -39,12 +53,19 @@ app = FastAPI(
)
# Add CORS middleware
# It's recommended to load the allowed origins from an environment variable
# for better security and flexibility across different environments.
allowed_origins_str = os.getenv("ALLOWED_ORIGINS", "http://localhost:3000")
allowed_origins = [origin.strip() for origin in allowed_origins_str.split(",")]
logger.info(f"Allowed origins: {allowed_origins}")
app.add_middleware(
CORSMiddleware,
allow_origins=["*"], # Allows all origins
allow_origins=allowed_origins, # Restrict to specific origins
allow_credentials=True,
allow_methods=["*"], # Allows all methods
allow_headers=["*"], # Allows all headers
allow_methods=["GET", "POST", "OPTIONS"], # Use the configured list of methods
allow_headers=["*"], # Now allow all headers, but can be restricted further
)
graph = build_graph_with_memory()
@@ -52,6 +73,20 @@ graph = build_graph_with_memory()
@app.post("/api/chat/stream")
async def chat_stream(request: ChatRequest):
# Check if MCP server configuration is enabled
mcp_enabled = os.getenv("ENABLE_MCP_SERVER_CONFIGURATION", "false").lower() in [
"true",
"1",
"yes",
]
# Validate MCP settings if provided
if request.mcp_settings and not mcp_enabled:
raise HTTPException(
status_code=403,
detail="MCP server configuration is disabled. Set ENABLE_MCP_SERVER_CONFIGURATION=true to enable MCP features.",
)
thread_id = request.thread_id
if thread_id == "__default__":
thread_id = str(uuid4())
@@ -59,26 +94,34 @@ async def chat_stream(request: ChatRequest):
_astream_workflow_generator(
request.model_dump()["messages"],
thread_id,
request.resources,
request.max_plan_iterations,
request.max_step_num,
request.max_search_results,
request.auto_accepted_plan,
request.interrupt_feedback,
request.mcp_settings,
request.mcp_settings if mcp_enabled else {},
request.enable_background_investigation,
request.report_style,
request.enable_deep_thinking,
),
media_type="text/event-stream",
)
async def _astream_workflow_generator(
messages: List[ChatMessage],
messages: List[dict],
thread_id: str,
resources: List[Resource],
max_plan_iterations: int,
max_step_num: int,
max_search_results: int,
auto_accepted_plan: bool,
interrupt_feedback: str,
mcp_settings: dict,
enable_background_investigation,
enable_background_investigation: bool,
report_style: ReportStyle,
enable_deep_thinking: bool,
):
input_ = {
"messages": messages,
@@ -88,6 +131,7 @@ async def _astream_workflow_generator(
"observations": [],
"auto_accepted_plan": auto_accepted_plan,
"enable_background_investigation": enable_background_investigation,
"research_topic": messages[-1]["content"] if messages else "",
}
if not auto_accepted_plan and interrupt_feedback:
resume_msg = f"[{interrupt_feedback}]"
@@ -99,9 +143,13 @@ async def _astream_workflow_generator(
input_,
config={
"thread_id": thread_id,
"resources": resources,
"max_plan_iterations": max_plan_iterations,
"max_step_num": max_step_num,
"max_search_results": max_search_results,
"mcp_settings": mcp_settings,
"report_style": report_style.value,
"enable_deep_thinking": enable_deep_thinking,
},
stream_mode=["messages", "updates"],
subgraphs=True,
@@ -124,15 +172,23 @@ async def _astream_workflow_generator(
)
continue
message_chunk, message_metadata = cast(
tuple[AIMessageChunk, dict[str, any]], event_data
tuple[BaseMessage, dict[str, any]], event_data
)
# Handle empty agent tuple gracefully
agent_name = "unknown"
if agent and len(agent) > 0:
agent_name = agent[0].split(":")[0] if ":" in agent[0] else agent[0]
event_stream_message: dict[str, any] = {
"thread_id": thread_id,
"agent": agent[0].split(":")[0],
"agent": agent_name,
"id": message_chunk.id,
"role": "assistant",
"content": message_chunk.content,
}
if message_chunk.additional_kwargs.get("reasoning_content"):
event_stream_message["reasoning_content"] = message_chunk.additional_kwargs[
"reasoning_content"
]
if message_chunk.response_metadata.get("finish_reason"):
event_stream_message["finish_reason"] = message_chunk.response_metadata.get(
"finish_reason"
@@ -141,7 +197,7 @@ async def _astream_workflow_generator(
# Tool Message - Return the result of the tool call
event_stream_message["tool_call_id"] = message_chunk.tool_call_id
yield _make_event("tool_call_result", event_stream_message)
else:
elif isinstance(message_chunk, AIMessageChunk):
# AI Message - Raw message tokens
if message_chunk.tool_calls:
# AI Message - Tool Call
@@ -170,17 +226,16 @@ def _make_event(event_type: str, data: dict[str, any]):
@app.post("/api/tts")
async def text_to_speech(request: TTSRequest):
"""Convert text to speech using volcengine TTS API."""
app_id = os.getenv("VOLCENGINE_TTS_APPID", "")
if not app_id:
raise HTTPException(status_code=400, detail="VOLCENGINE_TTS_APPID is not set")
access_token = os.getenv("VOLCENGINE_TTS_ACCESS_TOKEN", "")
if not access_token:
raise HTTPException(
status_code=400, detail="VOLCENGINE_TTS_ACCESS_TOKEN is not set"
)
try:
app_id = os.getenv("VOLCENGINE_TTS_APPID", "")
if not app_id:
raise HTTPException(
status_code=400, detail="VOLCENGINE_TTS_APPID is not set"
)
access_token = os.getenv("VOLCENGINE_TTS_ACCESS_TOKEN", "")
if not access_token:
raise HTTPException(
status_code=400, detail="VOLCENGINE_TTS_ACCESS_TOKEN is not set"
)
cluster = os.getenv("VOLCENGINE_TTS_CLUSTER", "volcano_tts")
voice_type = os.getenv("VOLCENGINE_TTS_VOICE_TYPE", "BV700_V2_streaming")
@@ -218,9 +273,10 @@ async def text_to_speech(request: TTSRequest):
)
},
)
except Exception as e:
logger.exception(f"Error in TTS endpoint: {str(e)}")
raise HTTPException(status_code=500, detail=str(e))
raise HTTPException(status_code=500, detail=INTERNAL_SERVER_ERROR_DETAIL)
@app.post("/api/podcast/generate")
@@ -234,7 +290,7 @@ async def generate_podcast(request: GeneratePodcastRequest):
return Response(content=audio_bytes, media_type="audio/mp3")
except Exception as e:
logger.exception(f"Error occurred during podcast generation: {str(e)}")
raise HTTPException(status_code=500, detail=str(e))
raise HTTPException(status_code=500, detail=INTERNAL_SERVER_ERROR_DETAIL)
@app.post("/api/ppt/generate")
@@ -253,13 +309,14 @@ async def generate_ppt(request: GeneratePPTRequest):
)
except Exception as e:
logger.exception(f"Error occurred during ppt generation: {str(e)}")
raise HTTPException(status_code=500, detail=str(e))
raise HTTPException(status_code=500, detail=INTERNAL_SERVER_ERROR_DETAIL)
@app.post("/api/prose/generate")
async def generate_prose(request: GenerateProseRequest):
try:
logger.info(f"Generating prose for prompt: {request.prompt}")
sanitized_prompt = request.prompt.replace("\r\n", "").replace("\n", "")
logger.info(f"Generating prose for prompt: {sanitized_prompt}")
workflow = build_prose_graph()
events = workflow.astream(
{
@@ -276,12 +333,63 @@ async def generate_prose(request: GenerateProseRequest):
)
except Exception as e:
logger.exception(f"Error occurred during prose generation: {str(e)}")
raise HTTPException(status_code=500, detail=str(e))
raise HTTPException(status_code=500, detail=INTERNAL_SERVER_ERROR_DETAIL)
@app.post("/api/prompt/enhance")
async def enhance_prompt(request: EnhancePromptRequest):
try:
sanitized_prompt = request.prompt.replace("\r\n", "").replace("\n", "")
logger.info(f"Enhancing prompt: {sanitized_prompt}")
# Convert string report_style to ReportStyle enum
report_style = None
if request.report_style:
try:
# Handle both uppercase and lowercase input
style_mapping = {
"ACADEMIC": ReportStyle.ACADEMIC,
"POPULAR_SCIENCE": ReportStyle.POPULAR_SCIENCE,
"NEWS": ReportStyle.NEWS,
"SOCIAL_MEDIA": ReportStyle.SOCIAL_MEDIA,
}
report_style = style_mapping.get(
request.report_style.upper(), ReportStyle.ACADEMIC
)
except Exception:
# If invalid style, default to ACADEMIC
report_style = ReportStyle.ACADEMIC
else:
report_style = ReportStyle.ACADEMIC
workflow = build_prompt_enhancer_graph()
final_state = workflow.invoke(
{
"prompt": request.prompt,
"context": request.context,
"report_style": report_style,
}
)
return {"result": final_state["output"]}
except Exception as e:
logger.exception(f"Error occurred during prompt enhancement: {str(e)}")
raise HTTPException(status_code=500, detail=INTERNAL_SERVER_ERROR_DETAIL)
@app.post("/api/mcp/server/metadata", response_model=MCPServerMetadataResponse)
async def mcp_server_metadata(request: MCPServerMetadataRequest):
"""Get information about an MCP server."""
# Check if MCP server configuration is enabled
if os.getenv("ENABLE_MCP_SERVER_CONFIGURATION", "false").lower() not in [
"true",
"1",
"yes",
]:
raise HTTPException(
status_code=403,
detail="MCP server configuration is disabled. Set ENABLE_MCP_SERVER_CONFIGURATION=true to enable MCP features.",
)
try:
# Set default timeout with a longer value for this endpoint
timeout = 300 # Default to 300 seconds for this endpoint
@@ -312,7 +420,29 @@ async def mcp_server_metadata(request: MCPServerMetadataRequest):
return response
except Exception as e:
if not isinstance(e, HTTPException):
logger.exception(f"Error in MCP server metadata endpoint: {str(e)}")
raise HTTPException(status_code=500, detail=str(e))
raise
logger.exception(f"Error in MCP server metadata endpoint: {str(e)}")
raise HTTPException(status_code=500, detail=INTERNAL_SERVER_ERROR_DETAIL)
@app.get("/api/rag/config", response_model=RAGConfigResponse)
async def rag_config():
"""Get the config of the RAG."""
return RAGConfigResponse(provider=SELECTED_RAG_PROVIDER)
@app.get("/api/rag/resources", response_model=RAGResourcesResponse)
async def rag_resources(request: Annotated[RAGResourceRequest, Query()]):
"""Get the resources of the RAG."""
retriever = build_retriever()
if retriever:
return RAGResourcesResponse(resources=retriever.list_resources(request.query))
return RAGResourcesResponse(resources=[])
@app.get("/api/config", response_model=ConfigResponse)
async def config():
"""Get the config of the server."""
return ConfigResponse(
rag=RAGConfigResponse(provider=SELECTED_RAG_PROVIDER),
models=get_configured_llm_models(),
)
+25
View File
@@ -5,6 +5,9 @@ from typing import List, Optional, Union
from pydantic import BaseModel, Field
from src.rag.retriever import Resource
from src.config.report_style import ReportStyle
class ContentItem(BaseModel):
type: str = Field(..., description="The type of content (text, image, etc.)")
@@ -28,6 +31,9 @@ class ChatRequest(BaseModel):
messages: Optional[List[ChatMessage]] = Field(
[], description="History of messages between the user and the assistant"
)
resources: Optional[List[Resource]] = Field(
[], description="Resources to be used for the research"
)
debug: Optional[bool] = Field(False, description="Whether to enable debug logging")
thread_id: Optional[str] = Field(
"__default__", description="A specific conversation identifier"
@@ -38,6 +44,9 @@ class ChatRequest(BaseModel):
max_step_num: Optional[int] = Field(
3, description="The maximum number of steps in a plan"
)
max_search_results: Optional[int] = Field(
3, description="The maximum number of search results"
)
auto_accepted_plan: Optional[bool] = Field(
False, description="Whether to automatically accept the plan"
)
@@ -50,6 +59,12 @@ class ChatRequest(BaseModel):
enable_background_investigation: Optional[bool] = Field(
True, description="Whether to get background investigation before plan"
)
report_style: Optional[ReportStyle] = Field(
ReportStyle.ACADEMIC, description="The style of the report"
)
enable_deep_thinking: Optional[bool] = Field(
False, description="Whether to enable deep thinking"
)
class TTSRequest(BaseModel):
@@ -82,3 +97,13 @@ class GenerateProseRequest(BaseModel):
command: Optional[str] = Field(
"", description="The user custom command of the prose writer"
)
class EnhancePromptRequest(BaseModel):
prompt: str = Field(..., description="The original prompt to enhance")
context: Optional[str] = Field(
"", description="Additional context about the intended use"
)
report_style: Optional[str] = Field(
"academic", description="The style of the report"
)
+13
View File
@@ -0,0 +1,13 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from pydantic import BaseModel, Field
from src.server.rag_request import RAGConfigResponse
class ConfigResponse(BaseModel):
"""Response model for server config."""
rag: RAGConfigResponse = Field(..., description="The config of the RAG")
models: dict[str, list[str]] = Field(..., description="The configured models")
+1 -1
View File
@@ -3,7 +3,7 @@
import logging
from datetime import timedelta
from typing import Any, Dict, List, Optional, Tuple
from typing import Any, Dict, List, Optional
from fastapi import HTTPException
from mcp import ClientSession, StdioServerParameters
+28
View File
@@ -0,0 +1,28 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from pydantic import BaseModel, Field
from src.rag.retriever import Resource
class RAGConfigResponse(BaseModel):
"""Response model for RAG config."""
provider: str | None = Field(
None, description="The provider of the RAG, default is ragflow"
)
class RAGResourceRequest(BaseModel):
"""Request model for RAG resource."""
query: str | None = Field(
None, description="The query of the resource need to be searched"
)
class RAGResourcesResponse(BaseModel):
"""Response model for RAG resources."""
resources: list[Resource] = Field(..., description="The resources of the RAG")
+4 -20
View File
@@ -1,32 +1,16 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import os
from .crawl import crawl_tool
from .python_repl import python_repl_tool
from .search import (
tavily_search_tool,
duckduckgo_search_tool,
brave_search_tool,
arxiv_search_tool,
)
from .retriever import get_retriever_tool
from .search import get_web_search_tool
from .tts import VolcengineTTS
from src.config import SELECTED_SEARCH_ENGINE, SearchEngine
# Map search engine names to their respective tools
search_tool_mappings = {
SearchEngine.TAVILY.value: tavily_search_tool,
SearchEngine.DUCKDUCKGO.value: duckduckgo_search_tool,
SearchEngine.BRAVE_SEARCH.value: brave_search_tool,
SearchEngine.ARXIV.value: arxiv_search_tool,
}
web_search_tool = search_tool_mappings.get(SELECTED_SEARCH_ENGINE, tavily_search_tool)
__all__ = [
"crawl_tool",
"web_search_tool",
"python_repl_tool",
"get_web_search_tool",
"get_retriever_tool",
"VolcengineTTS",
]
+62
View File
@@ -0,0 +1,62 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import logging
from typing import List, Optional, Type
from langchain_core.tools import BaseTool
from langchain_core.callbacks import (
AsyncCallbackManagerForToolRun,
CallbackManagerForToolRun,
)
from pydantic import BaseModel, Field
from src.config.tools import SELECTED_RAG_PROVIDER
from src.rag import Document, Retriever, Resource, build_retriever
logger = logging.getLogger(__name__)
class RetrieverInput(BaseModel):
keywords: str = Field(description="search keywords to look up")
class RetrieverTool(BaseTool):
name: str = "local_search_tool"
description: str = (
"Useful for retrieving information from the file with `rag://` uri prefix, it should be higher priority than the web search or writing code. Input should be a search keywords."
)
args_schema: Type[BaseModel] = RetrieverInput
retriever: Retriever = Field(default_factory=Retriever)
resources: list[Resource] = Field(default_factory=list)
def _run(
self,
keywords: str,
run_manager: Optional[CallbackManagerForToolRun] = None,
) -> list[Document]:
logger.info(
f"Retriever tool query: {keywords}", extra={"resources": self.resources}
)
documents = self.retriever.query_relevant_documents(keywords, self.resources)
if not documents:
return "No results found from the local knowledge base."
return [doc.to_dict() for doc in documents]
async def _arun(
self,
keywords: str,
run_manager: Optional[AsyncCallbackManagerForToolRun] = None,
) -> list[Document]:
return self._run(keywords, run_manager.get_sync())
def get_retriever_tool(resources: List[Resource]) -> RetrieverTool | None:
if not resources:
return None
logger.info(f"create retriever tool: {SELECTED_RAG_PROVIDER}")
retriever = build_retriever()
if not retriever:
return None
return RetrieverTool(retriever=retriever, resources=resources)
+57 -38
View File
@@ -1,15 +1,16 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import json
import logging
import os
from typing import List, Optional
from langchain_community.tools import BraveSearch, DuckDuckGoSearchResults
from langchain_community.tools.arxiv import ArxivQueryRun
from langchain_community.utilities import ArxivAPIWrapper, BraveSearchWrapper
from src.config import SEARCH_MAX_RESULTS, SearchEngine
from src.config import SearchEngine, SELECTED_SEARCH_ENGINE
from src.config import load_yaml_config
from src.tools.tavily_search.tavily_search_results_with_images import (
TavilySearchResultsWithImages,
)
@@ -18,44 +19,62 @@ from src.tools.decorators import create_logged_tool
logger = logging.getLogger(__name__)
# Create logged versions of the search tools
LoggedTavilySearch = create_logged_tool(TavilySearchResultsWithImages)
if os.getenv("SEARCH_API", "") == SearchEngine.TAVILY.value:
tavily_search_tool = LoggedTavilySearch(
name="web_search",
max_results=SEARCH_MAX_RESULTS,
include_raw_content=True,
include_images=True,
include_image_descriptions=True,
)
else:
tavily_search_tool = None
LoggedDuckDuckGoSearch = create_logged_tool(DuckDuckGoSearchResults)
duckduckgo_search_tool = LoggedDuckDuckGoSearch(
name="web_search", max_results=SEARCH_MAX_RESULTS
)
LoggedBraveSearch = create_logged_tool(BraveSearch)
brave_search_tool = LoggedBraveSearch(
name="web_search",
search_wrapper=BraveSearchWrapper(
api_key=os.getenv("BRAVE_SEARCH_API_KEY", ""),
search_kwargs={"count": SEARCH_MAX_RESULTS},
),
)
LoggedArxivSearch = create_logged_tool(ArxivQueryRun)
arxiv_search_tool = LoggedArxivSearch(
name="web_search",
api_wrapper=ArxivAPIWrapper(
top_k_results=SEARCH_MAX_RESULTS,
load_max_docs=SEARCH_MAX_RESULTS,
load_all_available_meta=True,
),
)
if __name__ == "__main__":
results = LoggedDuckDuckGoSearch(
name="web_search", max_results=SEARCH_MAX_RESULTS, output_format="list"
).invoke("cute panda")
print(json.dumps(results, indent=2, ensure_ascii=False))
def get_search_config():
config = load_yaml_config("conf.yaml")
search_config = config.get("SEARCH_ENGINE", {})
return search_config
# Get the selected search tool
def get_web_search_tool(max_search_results: int):
search_config = get_search_config()
if SELECTED_SEARCH_ENGINE == SearchEngine.TAVILY.value:
# Only get and apply include/exclude domains for Tavily
include_domains: Optional[List[str]] = search_config.get("include_domains", [])
exclude_domains: Optional[List[str]] = search_config.get("exclude_domains", [])
logger.info(
f"Tavily search configuration loaded: include_domains={include_domains}, exclude_domains={exclude_domains}"
)
return LoggedTavilySearch(
name="web_search",
max_results=max_search_results,
include_raw_content=True,
include_images=True,
include_image_descriptions=True,
include_domains=include_domains,
exclude_domains=exclude_domains,
)
elif SELECTED_SEARCH_ENGINE == SearchEngine.DUCKDUCKGO.value:
return LoggedDuckDuckGoSearch(
name="web_search",
num_results=max_search_results,
)
elif SELECTED_SEARCH_ENGINE == SearchEngine.BRAVE_SEARCH.value:
return LoggedBraveSearch(
name="web_search",
search_wrapper=BraveSearchWrapper(
api_key=os.getenv("BRAVE_SEARCH_API_KEY", ""),
search_kwargs={"count": max_search_results},
),
)
elif SELECTED_SEARCH_ENGINE == SearchEngine.ARXIV.value:
return LoggedArxivSearch(
name="web_search",
api_wrapper=ArxivAPIWrapper(
top_k_results=max_search_results,
load_max_docs=max_search_results,
load_all_available_meta=True,
),
)
else:
raise ValueError(f"Unsupported search engine: {SELECTED_SEARCH_ENGINE}")
@@ -1,3 +1,7 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import json
from typing import Dict, List, Optional
@@ -70,7 +74,7 @@ class EnhancedTavilySearchAPIWrapper(OriginalTavilySearchAPIWrapper):
"include_images": include_images,
"include_image_descriptions": include_image_descriptions,
}
async with aiohttp.ClientSession() as session:
async with aiohttp.ClientSession(trust_env=True) as session:
async with session.post(f"{TAVILY_API_URL}/search", json=params) as res:
if res.status == 200:
data = await res.text()
@@ -107,9 +111,3 @@ class EnhancedTavilySearchAPIWrapper(OriginalTavilySearchAPIWrapper):
}
clean_results.append(clean_result)
return clean_results
if __name__ == "__main__":
wrapper = EnhancedTavilySearchAPIWrapper()
results = wrapper.raw_results("cute panda", include_images=True)
print(json.dumps(results, indent=2, ensure_ascii=False))
@@ -1,3 +1,6 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import json
from typing import Dict, List, Optional, Tuple, Union
+3 -2
View File
@@ -102,7 +102,8 @@ class VolcengineTTS:
}
try:
logger.debug(f"Sending TTS request for text: {text[:50]}...")
sanitized_text = text.replace("\r\n", "").replace("\n", "")
logger.debug(f"Sending TTS request for text: {sanitized_text[:50]}...")
response = requests.post(
self.api_url, json.dumps(request_json), headers=self.header
)
@@ -128,4 +129,4 @@ class VolcengineTTS:
except Exception as e:
logger.exception(f"Error in TTS API call: {str(e)}")
return {"success": False, "error": str(e), "audio_data": None}
return {"success": False, "error": "TTS API call error", "audio_data": None}
+11 -15
View File
@@ -19,21 +19,17 @@ def repair_json_output(content: str) -> str:
str: Repaired JSON string, or original content if not JSON
"""
content = content.strip()
if content.startswith(("{", "[")) or "```json" in content or "```ts" in content:
try:
# If content is wrapped in ```json code block, extract the JSON part
if content.startswith("```json"):
content = content.removeprefix("```json")
if content.startswith("```ts"):
content = content.removeprefix("```ts")
try:
# Try to repair and parse JSON
repaired_content = json_repair.loads(content)
if not isinstance(repaired_content, dict) and not isinstance(
repaired_content, list
):
logger.warning("Repaired content is not a valid JSON object or array.")
return content
content = json.dumps(repaired_content, ensure_ascii=False)
except Exception as e:
logger.warning(f"JSON repair failed: {e}")
if content.endswith("```"):
content = content.removesuffix("```")
# Try to repair and parse JSON
repaired_content = json_repair.loads(content)
return json.dumps(repaired_content, ensure_ascii=False)
except Exception as e:
logger.warning(f"JSON repair failed: {e}")
return content
-1
View File
@@ -1,7 +1,6 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import asyncio
import logging
from src.graph import build_graph
+24
View File
@@ -0,0 +1,24 @@
#!/usr/bin/env python3
"""
This script manually patches sys.modules to fix the LLM import issue
so that tests can run without requiring LLM configuration.
"""
import sys
from unittest.mock import MagicMock
# Create mocks
mock_llm = MagicMock()
mock_llm.invoke.return_value = "Mock LLM response"
# Create a mock module for llm.py
mock_llm_module = MagicMock()
mock_llm_module.get_llm_by_type = lambda llm_type: mock_llm
mock_llm_module.basic_llm = mock_llm
mock_llm_module._create_llm_use_conf = lambda llm_type, conf: mock_llm
# Set the mock module
sys.modules["src.llms.llm"] = mock_llm_module
print("Successfully patched LLM module. You can now run your tests.")
print("Example: uv run pytest tests/test_types.py -v")
-1
View File
@@ -1,7 +1,6 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from src.crawler import Crawler
File diff suppressed because it is too large Load Diff
@@ -1,7 +1,6 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from src.tools.python_repl import python_repl_tool
+37 -2
View File
@@ -101,8 +101,43 @@ def test_current_time_format():
messages = apply_prompt_template("coder", test_state)
system_content = messages[0]["content"]
# Time format should be like: Mon Jan 01 2024 12:34:56 +0000
time_format = r"\w{3} \w{3} \d{2} \d{4} \d{2}:\d{2}:\d{2}"
assert any(
line.strip().startswith("CURRENT_TIME:") for line in system_content.split("\n")
)
def test_apply_prompt_template_reporter():
"""Test reporter template rendering with different styles and locale"""
test_state_news = {
"messages": [],
"task": "test reporter task",
"workspace_context": "test reporter context",
"report_style": "news",
"locale": "en-US",
}
messages_news = apply_prompt_template("reporter", test_state_news)
system_content_news = messages_news[0]["content"]
assert "NBC News" in system_content_news
test_state_social_media_en = {
"messages": [],
"task": "test reporter task",
"workspace_context": "test reporter context",
"report_style": "social_media",
"locale": "en-US",
}
messages_default = apply_prompt_template("reporter", test_state_social_media_en)
system_content_default = messages_default[0]["content"]
assert "Twitter/X" in system_content_default
test_state_social_media_cn = {
"messages": [],
"task": "test reporter task",
"workspace_context": "test reporter context",
"report_style": "social_media",
"locale": "zh-CN",
}
messages_cn = apply_prompt_template("reporter", test_state_social_media_cn)
system_content_cn = messages_cn[0]["content"]
assert "小红书" in system_content_cn
+18 -2
View File
@@ -2,9 +2,7 @@
# SPDX-License-Identifier: MIT
import json
import pytest
from unittest.mock import patch, MagicMock
import uuid
import base64
from src.tools.tts import VolcengineTTS
@@ -229,3 +227,21 @@ class TestVolcengineTTS:
args, kwargs = mock_post.call_args
request_json = json.loads(args[1])
assert request_json["user"]["uid"] == str(mock_uuid_value)
@patch("src.tools.tts.requests.post")
def test_text_to_speech_request_exception(self, mock_post):
"""Test error handling when requests.post raises an exception."""
# Mock requests.post to raise an exception
mock_post.side_effect = Exception("Network error")
# Create TTS client
tts = VolcengineTTS(
appid="test_appid",
access_token="test_token",
)
# Call the method
result = tts.text_to_speech("Hello, world!")
# Verify the result
assert result["success"] is False
# The TTS error is caught and returned as a string
assert result["error"] == "TTS API call error"
assert result["audio_data"] is None
+133
View File
@@ -0,0 +1,133 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import sys
import os
from typing import Annotated
# Import MessagesState directly from langgraph rather than through our application
from langgraph.graph import MessagesState
# Create stub versions of Plan/Step/StepType to avoid dependencies
class StepType:
RESEARCH = "research"
PROCESSING = "processing"
class Step:
def __init__(self, need_search, title, description, step_type):
self.need_search = need_search
self.title = title
self.description = description
self.step_type = step_type
class Plan:
def __init__(self, locale, has_enough_context, thought, title, steps):
self.locale = locale
self.has_enough_context = has_enough_context
self.thought = thought
self.title = title
self.steps = steps
# Import the actual State class by loading the module directly
# This avoids the cascade of imports that would normally happen
def load_state_class():
# Get the absolute path to the types.py file
src_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "src"))
types_path = os.path.join(src_dir, "graph", "types.py")
# Create a namespace for the module
import types
module_name = "src.graph.types_direct"
spec = types.ModuleType(module_name)
# Add the module to sys.modules to avoid import loops
sys.modules[module_name] = spec
# Set up the namespace with required imports
spec.__dict__["operator"] = __import__("operator")
spec.__dict__["Annotated"] = Annotated
spec.__dict__["MessagesState"] = MessagesState
spec.__dict__["Plan"] = Plan
# Execute the module code
with open(types_path, "r") as f:
module_code = f.read()
exec(module_code, spec.__dict__)
# Return the State class
return spec.State
# Load the actual State class
State = load_state_class()
def test_state_initialization():
"""Test that State class has correct default attribute definitions."""
# Test that the class has the expected attribute definitions
assert State.locale == "en-US"
assert State.observations == []
assert State.plan_iterations == 0
assert State.current_plan is None
assert State.final_report == ""
assert State.auto_accepted_plan is False
assert State.enable_background_investigation is True
assert State.background_investigation_results is None
# Verify state initialization
state = State(messages=[])
assert "messages" in state
# Without explicitly passing attributes, they're not in the state
assert "locale" not in state
assert "observations" not in state
def test_state_with_custom_values():
"""Test that State can be initialized with custom values."""
test_step = Step(
need_search=True,
title="Test Step",
description="Step description",
step_type=StepType.RESEARCH,
)
test_plan = Plan(
locale="en-US",
has_enough_context=False,
thought="Test thought",
title="Test Plan",
steps=[test_step],
)
# Initialize state with custom values and required messages field
state = State(
messages=[],
locale="fr-FR",
observations=["Observation 1"],
plan_iterations=2,
current_plan=test_plan,
final_report="Test report",
auto_accepted_plan=True,
enable_background_investigation=False,
background_investigation_results="Test results",
)
# Access state keys - these are explicitly initialized
assert state["locale"] == "fr-FR"
assert state["observations"] == ["Observation 1"]
assert state["plan_iterations"] == 2
assert state["current_plan"].title == "Test Plan"
assert state["current_plan"].thought == "Test thought"
assert len(state["current_plan"].steps) == 1
assert state["current_plan"].steps[0].title == "Test Step"
assert state["final_report"] == "Test report"
assert state["auto_accepted_plan"] is True
assert state["enable_background_investigation"] is False
assert state["background_investigation_results"] == "Test results"
+90
View File
@@ -0,0 +1,90 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import sys
import types
from src.config.configuration import Configuration
# Patch sys.path so relative import works
# Patch Resource for import
mock_resource = type("Resource", (), {})
# Patch src.rag.retriever.Resource for import
module_name = "src.rag.retriever"
if module_name not in sys.modules:
retriever_mod = types.ModuleType(module_name)
retriever_mod.Resource = mock_resource
sys.modules[module_name] = retriever_mod
# Relative import of Configuration
def test_default_configuration():
config = Configuration()
assert config.resources == []
assert config.max_plan_iterations == 1
assert config.max_step_num == 3
assert config.max_search_results == 3
assert config.mcp_settings is None
def test_from_runnable_config_with_config_dict(monkeypatch):
config_dict = {
"configurable": {
"max_plan_iterations": 5,
"max_step_num": 7,
"max_search_results": 10,
"mcp_settings": {"foo": "bar"},
}
}
config = Configuration.from_runnable_config(config_dict)
assert config.max_plan_iterations == 5
assert config.max_step_num == 7
assert config.max_search_results == 10
assert config.mcp_settings == {"foo": "bar"}
def test_from_runnable_config_with_env_override(monkeypatch):
monkeypatch.setenv("MAX_PLAN_ITERATIONS", "9")
monkeypatch.setenv("MAX_STEP_NUM", "11")
config_dict = {
"configurable": {
"max_plan_iterations": 2,
"max_step_num": 3,
"max_search_results": 4,
}
}
config = Configuration.from_runnable_config(config_dict)
# Environment variables take precedence and are strings
assert config.max_plan_iterations == "9"
assert config.max_step_num == "11"
assert config.max_search_results == 4 # not overridden
# Clean up
monkeypatch.delenv("MAX_PLAN_ITERATIONS")
monkeypatch.delenv("MAX_STEP_NUM")
def test_from_runnable_config_with_none_and_falsy(monkeypatch):
config_dict = {
"configurable": {
"max_plan_iterations": None,
"max_step_num": 0, # falsy, should be skipped
"max_search_results": "",
}
}
config = Configuration.from_runnable_config(config_dict)
# Should fall back to defaults for skipped/falsy values
assert config.max_plan_iterations == 1
assert config.max_step_num == 3
assert config.max_search_results == 3
def test_from_runnable_config_with_no_config():
config = Configuration.from_runnable_config()
assert config.max_plan_iterations == 1
assert config.max_step_num == 3
assert config.max_search_results == 3
assert config.resources == []
assert config.mcp_settings is None
+81
View File
@@ -0,0 +1,81 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import os
import tempfile
from src.config.loader import load_yaml_config, process_dict, replace_env_vars
def test_replace_env_vars_with_env(monkeypatch):
monkeypatch.setenv("TEST_ENV", "env_value")
assert replace_env_vars("$TEST_ENV") == "env_value"
def test_replace_env_vars_without_env(monkeypatch):
monkeypatch.delenv("NOT_SET_ENV", raising=False)
assert replace_env_vars("$NOT_SET_ENV") == "NOT_SET_ENV"
def test_replace_env_vars_non_string():
assert replace_env_vars(123) == 123
def test_replace_env_vars_regular_string():
assert replace_env_vars("no_env") == "no_env"
def test_process_dict_nested(monkeypatch):
monkeypatch.setenv("FOO", "bar")
config = {"a": "$FOO", "b": {"c": "$FOO", "d": 42, "e": "$NOT_SET_ENV"}}
processed = process_dict(config)
assert processed["a"] == "bar"
assert processed["b"]["c"] == "bar"
assert processed["b"]["d"] == 42
assert processed["b"]["e"] == "NOT_SET_ENV"
def test_process_dict_empty():
assert process_dict({}) == {}
def test_load_yaml_config_file_not_exist():
assert load_yaml_config("non_existent_file.yaml") == {}
def test_load_yaml_config(monkeypatch):
monkeypatch.setenv("MY_ENV", "my_value")
yaml_content = """
key1: value1
key2: $MY_ENV
nested:
key3: $MY_ENV
key4: 123
"""
with tempfile.NamedTemporaryFile("w+", delete=False) as tmp:
tmp.write(yaml_content)
tmp_path = tmp.name
try:
config = load_yaml_config(tmp_path)
assert config["key1"] == "value1"
assert config["key2"] == "my_value"
assert config["nested"]["key3"] == "my_value"
assert config["nested"]["key4"] == 123
finally:
os.remove(tmp_path)
def test_load_yaml_config_cache(monkeypatch):
monkeypatch.setenv("CACHE_ENV", "cache_value")
yaml_content = "foo: $CACHE_ENV"
with tempfile.NamedTemporaryFile("w+", delete=False) as tmp:
tmp.write(yaml_content)
tmp_path = tmp.name
try:
config1 = load_yaml_config(tmp_path)
config2 = load_yaml_config(tmp_path)
assert config1 is config2 # Should be cached (same object)
assert config1["foo"] == "cache_value"
finally:
os.remove(tmp_path)
+73
View File
@@ -0,0 +1,73 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from src.crawler.article import Article
class DummyMarkdownify:
"""A dummy markdownify replacement for patching if needed."""
@staticmethod
def markdownify(html):
return html
def test_to_markdown_includes_title(monkeypatch):
article = Article("Test Title", "<p>Hello <b>world</b>!</p>")
result = article.to_markdown(including_title=True)
assert result.startswith("# Test Title")
assert "Hello" in result
def test_to_markdown_excludes_title():
article = Article("Test Title", "<p>Hello <b>world</b>!</p>")
result = article.to_markdown(including_title=False)
assert not result.startswith("# Test Title")
assert "Hello" in result
def test_to_message_with_text_only():
article = Article("Test Title", "<p>Hello world!</p>")
article.url = "https://example.com/"
result = article.to_message()
assert isinstance(result, list)
assert any(item["type"] == "text" for item in result)
assert all("type" in item for item in result)
def test_to_message_with_image(monkeypatch):
html = '<p>Intro</p><img src="img/pic.png"/>'
article = Article("Title", html)
article.url = "https://host.com/path/"
# The markdownify library will convert <img> to markdown image syntax
result = article.to_message()
# Should have both text and image_url types
types = [item["type"] for item in result]
assert "image_url" in types
assert "text" in types
# Check that the image_url is correctly joined
image_items = [item for item in result if item["type"] == "image_url"]
assert image_items
assert image_items[0]["image_url"]["url"] == "https://host.com/path/img/pic.png"
def test_to_message_multiple_images():
html = '<p>Start</p><img src="a.png"/><p>Mid</p><img src="b.jpg"/>End'
article = Article("Title", html)
article.url = "http://x/"
result = article.to_message()
image_urls = [
item["image_url"]["url"] for item in result if item["type"] == "image_url"
]
assert "http://x/a.png" in image_urls
assert "http://x/b.jpg" in image_urls
text_items = [item for item in result if item["type"] == "text"]
assert any("Start" in item["text"] for item in text_items)
assert any("Mid" in item["text"] for item in text_items)
def test_to_message_handles_empty_html():
article = Article("Empty", "")
article.url = "http://test/"
result = article.to_message()
assert isinstance(result, list)
assert result[0]["type"] == "text"
+70
View File
@@ -0,0 +1,70 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import src.crawler as crawler_module
def test_crawler_sets_article_url(monkeypatch):
"""Test that the crawler sets the article.url field correctly."""
class DummyArticle:
def __init__(self):
self.url = None
def to_markdown(self):
return "# Dummy"
class DummyJinaClient:
def crawl(self, url, return_format=None):
return "<html>dummy</html>"
class DummyReadabilityExtractor:
def extract_article(self, html):
return DummyArticle()
monkeypatch.setattr("src.crawler.crawler.JinaClient", DummyJinaClient)
monkeypatch.setattr(
"src.crawler.crawler.ReadabilityExtractor", DummyReadabilityExtractor
)
crawler = crawler_module.Crawler()
url = "http://example.com"
article = crawler.crawl(url)
assert article.url == url
assert article.to_markdown() == "# Dummy"
def test_crawler_calls_dependencies(monkeypatch):
"""Test that Crawler calls JinaClient.crawl and ReadabilityExtractor.extract_article."""
calls = {}
class DummyJinaClient:
def crawl(self, url, return_format=None):
calls["jina"] = (url, return_format)
return "<html>dummy</html>"
class DummyReadabilityExtractor:
def extract_article(self, html):
calls["extractor"] = html
class DummyArticle:
url = None
def to_markdown(self):
return "# Dummy"
return DummyArticle()
monkeypatch.setattr("src.crawler.crawler.JinaClient", DummyJinaClient)
monkeypatch.setattr(
"src.crawler.crawler.ReadabilityExtractor", DummyReadabilityExtractor
)
crawler = crawler_module.Crawler()
url = "http://example.com"
crawler.crawl(url)
assert "jina" in calls
assert calls["jina"][0] == url
assert calls["jina"][1] == "html"
assert "extractor" in calls
assert calls["extractor"] == "<html>dummy</html>"
+132
View File
@@ -0,0 +1,132 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from unittest.mock import MagicMock, patch
import importlib
import sys
import src.graph.builder as builder_mod
@pytest.fixture
def mock_state():
class Step:
def __init__(self, execution_res=None, step_type=None):
self.execution_res = execution_res
self.step_type = step_type
class Plan:
def __init__(self, steps):
self.steps = steps
return {
"Step": Step,
"Plan": Plan,
}
def test_continue_to_running_research_team_no_plan(mock_state):
state = {"current_plan": None}
assert builder_mod.continue_to_running_research_team(state) == "planner"
def test_continue_to_running_research_team_no_steps(mock_state):
state = {"current_plan": mock_state["Plan"](steps=[])}
assert builder_mod.continue_to_running_research_team(state) == "planner"
def test_continue_to_running_research_team_all_executed(mock_state):
Step = mock_state["Step"]
Plan = mock_state["Plan"]
steps = [Step(execution_res=True), Step(execution_res=True)]
state = {"current_plan": Plan(steps=steps)}
assert builder_mod.continue_to_running_research_team(state) == "planner"
def test_continue_to_running_research_team_next_researcher(mock_state):
Step = mock_state["Step"]
Plan = mock_state["Plan"]
steps = [
Step(execution_res=True),
Step(execution_res=None, step_type=builder_mod.StepType.RESEARCH),
]
state = {"current_plan": Plan(steps=steps)}
assert builder_mod.continue_to_running_research_team(state) == "researcher"
def test_continue_to_running_research_team_next_coder(mock_state):
Step = mock_state["Step"]
Plan = mock_state["Plan"]
steps = [
Step(execution_res=True),
Step(execution_res=None, step_type=builder_mod.StepType.PROCESSING),
]
state = {"current_plan": Plan(steps=steps)}
assert builder_mod.continue_to_running_research_team(state) == "coder"
def test_continue_to_running_research_team_next_coder_withresult(mock_state):
Step = mock_state["Step"]
Plan = mock_state["Plan"]
steps = [
Step(execution_res=True),
Step(execution_res=True, step_type=builder_mod.StepType.PROCESSING),
]
state = {"current_plan": Plan(steps=steps)}
assert builder_mod.continue_to_running_research_team(state) == "planner"
def test_continue_to_running_research_team_default_planner(mock_state):
Step = mock_state["Step"]
Plan = mock_state["Plan"]
steps = [Step(execution_res=True), Step(execution_res=None, step_type=None)]
state = {"current_plan": Plan(steps=steps)}
assert builder_mod.continue_to_running_research_team(state) == "planner"
@patch("src.graph.builder.StateGraph")
def test_build_base_graph_adds_nodes_and_edges(MockStateGraph):
mock_builder = MagicMock()
MockStateGraph.return_value = mock_builder
builder_mod._build_base_graph()
# Check that all nodes and edges are added
assert mock_builder.add_edge.call_count >= 2
assert mock_builder.add_node.call_count >= 8
mock_builder.add_conditional_edges.assert_called_once()
@patch("src.graph.builder._build_base_graph")
@patch("src.graph.builder.MemorySaver")
def test_build_graph_with_memory_uses_memory(MockMemorySaver, mock_build_base_graph):
mock_builder = MagicMock()
mock_build_base_graph.return_value = mock_builder
mock_memory = MagicMock()
MockMemorySaver.return_value = mock_memory
builder_mod.build_graph_with_memory()
mock_builder.compile.assert_called_once_with(checkpointer=mock_memory)
@patch("src.graph.builder._build_base_graph")
def test_build_graph_without_memory(mock_build_base_graph):
mock_builder = MagicMock()
mock_build_base_graph.return_value = mock_builder
builder_mod.build_graph()
mock_builder.compile.assert_called_once_with()
def test_graph_is_compiled():
# The graph object should be the result of build_graph()
with patch("src.graph.builder._build_base_graph") as mock_base:
mock_builder = MagicMock()
mock_base.return_value = mock_builder
mock_builder.compile.return_value = "compiled_graph"
# reload the module to re-run the graph assignment
importlib.reload(sys.modules["src.graph.builder"])
assert builder_mod.graph is not None
+86
View File
@@ -0,0 +1,86 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from src.llms import llm
class DummyChatOpenAI:
def __init__(self, **kwargs):
self.kwargs = kwargs
def invoke(self, msg):
return f"Echo: {msg}"
@pytest.fixture(autouse=True)
def patch_chat_openai(monkeypatch):
monkeypatch.setattr(llm, "ChatOpenAI", DummyChatOpenAI)
@pytest.fixture
def dummy_conf():
return {
"BASIC_MODEL": {"api_key": "test_key", "base_url": "http://test"},
"REASONING_MODEL": {"api_key": "reason_key"},
"VISION_MODEL": {"api_key": "vision_key"},
}
def test_get_env_llm_conf(monkeypatch):
# Clear any existing environment variables that might interfere
monkeypatch.delenv("BASIC_MODEL__API_KEY", raising=False)
monkeypatch.delenv("BASIC_MODEL__BASE_URL", raising=False)
monkeypatch.delenv("BASIC_MODEL__MODEL", raising=False)
monkeypatch.setenv("BASIC_MODEL__API_KEY", "env_key")
monkeypatch.setenv("BASIC_MODEL__BASE_URL", "http://env")
conf = llm._get_env_llm_conf("basic")
assert conf["api_key"] == "env_key"
assert conf["base_url"] == "http://env"
def test_create_llm_use_conf_merges_env(monkeypatch, dummy_conf):
# Clear any existing environment variables that might interfere
monkeypatch.delenv("BASIC_MODEL__BASE_URL", raising=False)
monkeypatch.delenv("BASIC_MODEL__MODEL", raising=False)
monkeypatch.setenv("BASIC_MODEL__API_KEY", "env_key")
result = llm._create_llm_use_conf("basic", dummy_conf)
assert isinstance(result, DummyChatOpenAI)
assert result.kwargs["api_key"] == "env_key"
assert result.kwargs["base_url"] == "http://test"
def test_create_llm_use_conf_invalid_type(monkeypatch, dummy_conf):
# Clear any existing environment variables that might interfere
monkeypatch.delenv("BASIC_MODEL__API_KEY", raising=False)
monkeypatch.delenv("BASIC_MODEL__BASE_URL", raising=False)
monkeypatch.delenv("BASIC_MODEL__MODEL", raising=False)
with pytest.raises(ValueError):
llm._create_llm_use_conf("unknown", dummy_conf)
def test_create_llm_use_conf_empty_conf(monkeypatch):
# Clear any existing environment variables that might interfere
monkeypatch.delenv("BASIC_MODEL__API_KEY", raising=False)
monkeypatch.delenv("BASIC_MODEL__BASE_URL", raising=False)
monkeypatch.delenv("BASIC_MODEL__MODEL", raising=False)
with pytest.raises(ValueError):
llm._create_llm_use_conf("basic", {})
def test_get_llm_by_type_caches(monkeypatch, dummy_conf):
called = {}
def fake_load_yaml_config(path):
called["called"] = True
return dummy_conf
monkeypatch.setattr(llm, "load_yaml_config", fake_load_yaml_config)
llm._llm_cache.clear()
inst1 = llm.get_llm_by_type("basic")
inst2 = llm.get_llm_by_type("basic")
assert inst1 is inst2
assert called["called"]
+2
View File
@@ -0,0 +1,2 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
@@ -0,0 +1,2 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
@@ -0,0 +1,155 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from unittest.mock import patch, MagicMock
from src.prompt_enhancer.graph.builder import build_graph
from src.prompt_enhancer.graph.state import PromptEnhancerState
class TestBuildGraph:
"""Test cases for build_graph function."""
@patch("src.prompt_enhancer.graph.builder.StateGraph")
def test_build_graph_structure(self, mock_state_graph):
"""Test that build_graph creates the correct graph structure."""
mock_builder = MagicMock()
mock_compiled_graph = MagicMock()
mock_state_graph.return_value = mock_builder
mock_builder.compile.return_value = mock_compiled_graph
result = build_graph()
# Verify StateGraph was created with correct state type
mock_state_graph.assert_called_once_with(PromptEnhancerState)
# Verify entry point was set
mock_builder.set_entry_point.assert_called_once_with("enhancer")
# Verify finish point was set
mock_builder.set_finish_point.assert_called_once_with("enhancer")
# Verify graph was compiled
mock_builder.compile.assert_called_once()
# Verify return value
assert result == mock_compiled_graph
@patch("src.prompt_enhancer.graph.builder.StateGraph")
@patch("src.prompt_enhancer.graph.builder.prompt_enhancer_node")
def test_build_graph_node_function(self, mock_enhancer_node, mock_state_graph):
"""Test that the correct node function is added to the graph."""
mock_builder = MagicMock()
mock_compiled_graph = MagicMock()
mock_state_graph.return_value = mock_builder
mock_builder.compile.return_value = mock_compiled_graph
build_graph()
# Verify the correct node function was added
mock_builder.add_node.assert_called_once_with("enhancer", mock_enhancer_node)
def test_build_graph_returns_compiled_graph(self):
"""Test that build_graph returns a compiled graph object."""
with patch("src.prompt_enhancer.graph.builder.StateGraph") as mock_state_graph:
mock_builder = MagicMock()
mock_compiled_graph = MagicMock()
mock_state_graph.return_value = mock_builder
mock_builder.compile.return_value = mock_compiled_graph
result = build_graph()
assert result is mock_compiled_graph
@patch("src.prompt_enhancer.graph.builder.StateGraph")
def test_build_graph_call_sequence(self, mock_state_graph):
"""Test that build_graph calls methods in the correct sequence."""
mock_builder = MagicMock()
mock_compiled_graph = MagicMock()
mock_state_graph.return_value = mock_builder
mock_builder.compile.return_value = mock_compiled_graph
# Track call order
call_order = []
def track_add_node(*args, **kwargs):
call_order.append("add_node")
def track_set_entry_point(*args, **kwargs):
call_order.append("set_entry_point")
def track_set_finish_point(*args, **kwargs):
call_order.append("set_finish_point")
def track_compile(*args, **kwargs):
call_order.append("compile")
return mock_compiled_graph
mock_builder.add_node.side_effect = track_add_node
mock_builder.set_entry_point.side_effect = track_set_entry_point
mock_builder.set_finish_point.side_effect = track_set_finish_point
mock_builder.compile.side_effect = track_compile
build_graph()
# Verify the correct call sequence
expected_order = ["add_node", "set_entry_point", "set_finish_point", "compile"]
assert call_order == expected_order
def test_build_graph_integration(self):
"""Integration test to verify the graph can be built without mocking."""
# This test verifies that all imports and dependencies are correct
try:
graph = build_graph()
assert graph is not None
# The graph should be a compiled LangGraph object
assert hasattr(graph, "invoke") or hasattr(graph, "stream")
except ImportError as e:
pytest.skip(f"Skipping integration test due to missing dependencies: {e}")
except Exception as e:
# If there are configuration issues (like missing LLM config),
# we still consider the test successful if the graph structure is built
if "LLM" in str(e) or "configuration" in str(e).lower():
pytest.skip(
f"Skipping integration test due to configuration issues: {e}"
)
else:
raise
@patch("src.prompt_enhancer.graph.builder.StateGraph")
def test_build_graph_single_node_workflow(self, mock_state_graph):
"""Test that the graph is configured as a single-node workflow."""
mock_builder = MagicMock()
mock_compiled_graph = MagicMock()
mock_state_graph.return_value = mock_builder
mock_builder.compile.return_value = mock_compiled_graph
build_graph()
# Verify only one node is added
assert mock_builder.add_node.call_count == 1
# Verify entry and finish points are the same node
mock_builder.set_entry_point.assert_called_once_with("enhancer")
mock_builder.set_finish_point.assert_called_once_with("enhancer")
@patch("src.prompt_enhancer.graph.builder.StateGraph")
def test_build_graph_state_type(self, mock_state_graph):
"""Test that the graph is initialized with the correct state type."""
mock_builder = MagicMock()
mock_compiled_graph = MagicMock()
mock_state_graph.return_value = mock_builder
mock_builder.compile.return_value = mock_compiled_graph
build_graph()
# Verify StateGraph was initialized with PromptEnhancerState
args, kwargs = mock_state_graph.call_args
assert args[0] == PromptEnhancerState
@@ -0,0 +1,525 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from unittest.mock import patch, MagicMock
from langchain.schema import HumanMessage, SystemMessage
from src.prompt_enhancer.graph.enhancer_node import prompt_enhancer_node
from src.prompt_enhancer.graph.state import PromptEnhancerState
from src.config.report_style import ReportStyle
@pytest.fixture
def mock_llm():
"""Mock LLM that returns a test response."""
llm = MagicMock()
llm.invoke.return_value = MagicMock(
content="""Thoughts: LLM thinks a lot
<enhanced_prompt>
Enhanced test prompt
</enhanced_prompt>
"""
)
return llm
@pytest.fixture
def mock_llm_xml_with_whitespace():
"""Mock LLM that returns XML response with extra whitespace."""
llm = MagicMock()
llm.invoke.return_value = MagicMock(
content="""
Some thoughts here...
<enhanced_prompt>
Enhanced prompt with whitespace
</enhanced_prompt>
Additional content after XML
"""
)
return llm
@pytest.fixture
def mock_llm_xml_multiline():
"""Mock LLM that returns XML response with multiline content."""
llm = MagicMock()
llm.invoke.return_value = MagicMock(
content="""
<enhanced_prompt>
This is a multiline enhanced prompt
that spans multiple lines
and includes various formatting.
It should preserve the structure.
</enhanced_prompt>
"""
)
return llm
@pytest.fixture
def mock_llm_no_xml():
"""Mock LLM that returns response without XML tags."""
llm = MagicMock()
llm.invoke.return_value = MagicMock(
content="Enhanced Prompt: This is an enhanced prompt without XML tags"
)
return llm
@pytest.fixture
def mock_llm_malformed_xml():
"""Mock LLM that returns response with malformed XML."""
llm = MagicMock()
llm.invoke.return_value = MagicMock(
content="""
<enhanced_prompt>
This XML tag is not properly closed
<enhanced_prompt>
"""
)
return llm
@pytest.fixture
def mock_messages():
"""Mock messages returned by apply_prompt_template."""
return [
SystemMessage(content="System prompt template"),
HumanMessage(content="Test human message"),
]
class TestPromptEnhancerNode:
"""Test cases for prompt_enhancer_node function."""
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_basic_prompt_enhancement(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test basic prompt enhancement without context or report style."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
state = PromptEnhancerState(prompt="Write about AI")
result = prompt_enhancer_node(state)
# Verify LLM was called
mock_get_llm.assert_called_once_with("basic")
mock_llm.invoke.assert_called_once_with(mock_messages)
# Verify apply_prompt_template was called correctly
mock_apply_template.assert_called_once()
call_args = mock_apply_template.call_args
assert call_args[0][0] == "prompt_enhancer/prompt_enhancer"
assert "messages" in call_args[0][1]
assert "report_style" in call_args[0][1]
# Verify result
assert result == {"output": "Enhanced test prompt"}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_prompt_enhancement_with_report_style(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test prompt enhancement with report style."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
state = PromptEnhancerState(
prompt="Write about AI", report_style=ReportStyle.ACADEMIC
)
result = prompt_enhancer_node(state)
# Verify apply_prompt_template was called with report_style
mock_apply_template.assert_called_once()
call_args = mock_apply_template.call_args
assert call_args[0][0] == "prompt_enhancer/prompt_enhancer"
assert call_args[0][1]["report_style"] == ReportStyle.ACADEMIC
# Verify result
assert result == {"output": "Enhanced test prompt"}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_prompt_enhancement_with_context(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test prompt enhancement with additional context."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
state = PromptEnhancerState(
prompt="Write about AI", context="Focus on machine learning applications"
)
result = prompt_enhancer_node(state)
# Verify apply_prompt_template was called
mock_apply_template.assert_called_once()
call_args = mock_apply_template.call_args
# Check that the context was included in the human message
messages_arg = call_args[0][1]["messages"]
assert len(messages_arg) == 1
human_message = messages_arg[0]
assert isinstance(human_message, HumanMessage)
assert "Focus on machine learning applications" in human_message.content
assert result == {"output": "Enhanced test prompt"}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_error_handling(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test error handling when LLM call fails."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
# Mock LLM to raise an exception
mock_llm.invoke.side_effect = Exception("LLM error")
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
# Should return original prompt on error
assert result == {"output": "Test prompt"}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_template_error_handling(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test error handling when template application fails."""
mock_get_llm.return_value = mock_llm
# Mock apply_prompt_template to raise an exception
mock_apply_template.side_effect = Exception("Template error")
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
# Should return original prompt on error
assert result == {"output": "Test prompt"}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_prefix_removal(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test that common prefixes are removed from LLM response."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
# Test different prefixes that should be removed
test_cases = [
"Enhanced Prompt: This is the enhanced prompt",
"Enhanced prompt: This is the enhanced prompt",
"Here's the enhanced prompt: This is the enhanced prompt",
"Here is the enhanced prompt: This is the enhanced prompt",
"**Enhanced Prompt**: This is the enhanced prompt",
"**Enhanced prompt**: This is the enhanced prompt",
]
for response_with_prefix in test_cases:
mock_llm.invoke.return_value = MagicMock(content=response_with_prefix)
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
assert result == {"output": "This is the enhanced prompt"}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_whitespace_handling(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test that whitespace is properly stripped from LLM response."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
# Mock LLM response with extra whitespace
mock_llm.invoke.return_value = MagicMock(
content=" \n\n Enhanced prompt \n\n "
)
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
assert result == {"output": "Enhanced prompt"}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_xml_with_whitespace_handling(
self,
mock_get_llm,
mock_apply_template,
mock_llm_xml_with_whitespace,
mock_messages,
):
"""Test XML extraction with extra whitespace inside tags."""
mock_get_llm.return_value = mock_llm_xml_with_whitespace
mock_apply_template.return_value = mock_messages
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
assert result == {"output": "Enhanced prompt with whitespace"}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_xml_multiline_content(
self, mock_get_llm, mock_apply_template, mock_llm_xml_multiline, mock_messages
):
"""Test XML extraction with multiline content."""
mock_get_llm.return_value = mock_llm_xml_multiline
mock_apply_template.return_value = mock_messages
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
expected_output = """This is a multiline enhanced prompt
that spans multiple lines
and includes various formatting.
It should preserve the structure."""
assert result == {"output": expected_output}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_fallback_to_prefix_removal(
self, mock_get_llm, mock_apply_template, mock_llm_no_xml, mock_messages
):
"""Test fallback to prefix removal when no XML tags are found."""
mock_get_llm.return_value = mock_llm_no_xml
mock_apply_template.return_value = mock_messages
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
assert result == {"output": "This is an enhanced prompt without XML tags"}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_malformed_xml_fallback(
self, mock_get_llm, mock_apply_template, mock_llm_malformed_xml, mock_messages
):
"""Test handling of malformed XML tags."""
mock_get_llm.return_value = mock_llm_malformed_xml
mock_apply_template.return_value = mock_messages
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
# Should fall back to using the entire content since XML is malformed
expected_content = """<enhanced_prompt>
This XML tag is not properly closed
<enhanced_prompt>"""
assert result == {"output": expected_content}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_case_sensitive_prefix_removal(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test that prefix removal is case-sensitive."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
# Test case variations that should NOT be removed
test_cases = [
"ENHANCED PROMPT: This should not be removed",
"enhanced prompt: This should not be removed",
"Enhanced Prompt This should not be removed", # Missing colon
"Enhanced Prompt :: This should not be removed", # Double colon
]
for response_content in test_cases:
mock_llm.invoke.return_value = MagicMock(content=response_content)
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
# Should return the full content since prefix doesn't match exactly
assert result == {"output": response_content}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_prefix_with_extra_whitespace(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test prefix removal with extra whitespace after colon."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
test_cases = [
("Enhanced Prompt: This has extra spaces", "This has extra spaces"),
("Enhanced prompt:\t\tThis has tabs", "This has tabs"),
("Here's the enhanced prompt:\n\nThis has newlines", "This has newlines"),
]
for response_content, expected_output in test_cases:
mock_llm.invoke.return_value = MagicMock(content=response_content)
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
assert result == {"output": expected_output}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_xml_with_special_characters(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test XML extraction with special characters and symbols."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
special_content = """<enhanced_prompt>
Enhanced prompt with special chars: @#$%^&*()
Unicode: 🚀 💡
Quotes: "double" and 'single'
Backslashes: \\n \\t \\r
</enhanced_prompt>"""
mock_llm.invoke.return_value = MagicMock(content=special_content)
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
expected_output = """Enhanced prompt with special chars: @#$%^&*()
Unicode: 🚀 💡
Quotes: "double" and 'single'
Backslashes: \\n \\t \\r"""
assert result == {"output": expected_output}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_very_long_response(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test handling of very long LLM responses."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
# Create a very long response
long_content = "This is a very long enhanced prompt. " * 100
xml_response = f"<enhanced_prompt>\n{long_content}\n</enhanced_prompt>"
mock_llm.invoke.return_value = MagicMock(content=xml_response)
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
assert result == {"output": long_content.strip()}
assert len(result["output"]) > 1000 # Verify it's actually long
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_empty_response_content(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test handling of empty response content."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
mock_llm.invoke.return_value = MagicMock(content="")
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
assert result == {"output": ""}
@patch("src.prompt_enhancer.graph.enhancer_node.apply_prompt_template")
@patch("src.prompt_enhancer.graph.enhancer_node.get_llm_by_type")
@patch(
"src.prompt_enhancer.graph.enhancer_node.AGENT_LLM_MAP",
{"prompt_enhancer": "basic"},
)
def test_only_whitespace_response(
self, mock_get_llm, mock_apply_template, mock_llm, mock_messages
):
"""Test handling of response with only whitespace."""
mock_get_llm.return_value = mock_llm
mock_apply_template.return_value = mock_messages
mock_llm.invoke.return_value = MagicMock(content=" \n\n\t\t ")
state = PromptEnhancerState(prompt="Test prompt")
result = prompt_enhancer_node(state)
assert result == {"output": ""}
@@ -0,0 +1,107 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from src.prompt_enhancer.graph.state import PromptEnhancerState
from src.config.report_style import ReportStyle
def test_prompt_enhancer_state_creation():
"""Test that PromptEnhancerState can be created with required fields."""
state = PromptEnhancerState(
prompt="Test prompt", context=None, report_style=None, output=None
)
assert state["prompt"] == "Test prompt"
assert state["context"] is None
assert state["report_style"] is None
assert state["output"] is None
def test_prompt_enhancer_state_with_all_fields():
"""Test PromptEnhancerState with all fields populated."""
state = PromptEnhancerState(
prompt="Write about AI",
context="Additional context about AI research",
report_style=ReportStyle.ACADEMIC,
output="Enhanced prompt about AI research",
)
assert state["prompt"] == "Write about AI"
assert state["context"] == "Additional context about AI research"
assert state["report_style"] == ReportStyle.ACADEMIC
assert state["output"] == "Enhanced prompt about AI research"
def test_prompt_enhancer_state_minimal():
"""Test PromptEnhancerState with only required prompt field."""
state = PromptEnhancerState(prompt="Minimal prompt")
assert state["prompt"] == "Minimal prompt"
# Optional fields should not be present if not specified
assert "context" not in state
assert "report_style" not in state
assert "output" not in state
def test_prompt_enhancer_state_with_different_report_styles():
"""Test PromptEnhancerState with different ReportStyle values."""
styles = [
ReportStyle.ACADEMIC,
ReportStyle.POPULAR_SCIENCE,
ReportStyle.NEWS,
ReportStyle.SOCIAL_MEDIA,
]
for style in styles:
state = PromptEnhancerState(prompt="Test prompt", report_style=style)
assert state["report_style"] == style
def test_prompt_enhancer_state_update():
"""Test updating PromptEnhancerState fields."""
state = PromptEnhancerState(prompt="Original prompt")
# Update with new fields
state.update(
{
"context": "New context",
"report_style": ReportStyle.NEWS,
"output": "Enhanced output",
}
)
assert state["prompt"] == "Original prompt"
assert state["context"] == "New context"
assert state["report_style"] == ReportStyle.NEWS
assert state["output"] == "Enhanced output"
def test_prompt_enhancer_state_get_method():
"""Test using get() method on PromptEnhancerState."""
state = PromptEnhancerState(prompt="Test prompt", report_style=ReportStyle.ACADEMIC)
# Test get with existing keys
assert state.get("prompt") == "Test prompt"
assert state.get("report_style") == ReportStyle.ACADEMIC
# Test get with non-existing keys
assert state.get("context") is None
assert state.get("output") is None
assert state.get("nonexistent", "default") == "default"
def test_prompt_enhancer_state_type_annotations():
"""Test that the state accepts correct types."""
# This test ensures the TypedDict structure is working correctly
state = PromptEnhancerState(
prompt="Test prompt",
context="Test context",
report_style=ReportStyle.POPULAR_SCIENCE,
output="Test output",
)
# Verify types
assert isinstance(state["prompt"], str)
assert isinstance(state["context"], str)
assert isinstance(state["report_style"], ReportStyle)
assert isinstance(state["output"], str)
+155
View File
@@ -0,0 +1,155 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from unittest.mock import patch, MagicMock
from src.rag.ragflow import RAGFlowProvider, parse_uri
# Dummy classes to mock dependencies
class DummyResource:
def __init__(self, uri, title="", description=""):
self.uri = uri
self.title = title
self.description = description
class DummyChunk:
def __init__(self, content, similarity):
self.content = content
self.similarity = similarity
class DummyDocument:
def __init__(self, id, title, chunks=None):
self.id = id
self.title = title
self.chunks = chunks or []
# Patch imports in ragflow.py to use dummy classes
@pytest.fixture(autouse=True)
def patch_imports(monkeypatch):
import src.rag.ragflow as ragflow
ragflow.Resource = DummyResource
ragflow.Chunk = DummyChunk
ragflow.Document = DummyDocument
yield
def test_parse_uri_valid():
uri = "rag://dataset/123#abc"
dataset_id, document_id = parse_uri(uri)
assert dataset_id == "123"
assert document_id == "abc"
def test_parse_uri_invalid():
with pytest.raises(ValueError):
parse_uri("http://dataset/123#abc")
def test_init_env_vars(monkeypatch):
monkeypatch.setenv("RAGFLOW_API_URL", "http://api")
monkeypatch.setenv("RAGFLOW_API_KEY", "key")
monkeypatch.delenv("RAGFLOW_PAGE_SIZE", raising=False)
provider = RAGFlowProvider()
assert provider.api_url == "http://api"
assert provider.api_key == "key"
assert provider.page_size == 10
def test_init_page_size(monkeypatch):
monkeypatch.setenv("RAGFLOW_API_URL", "http://api")
monkeypatch.setenv("RAGFLOW_API_KEY", "key")
monkeypatch.setenv("RAGFLOW_PAGE_SIZE", "5")
provider = RAGFlowProvider()
assert provider.page_size == 5
def test_init_missing_env(monkeypatch):
monkeypatch.delenv("RAGFLOW_API_URL", raising=False)
monkeypatch.setenv("RAGFLOW_API_KEY", "key")
with pytest.raises(ValueError):
RAGFlowProvider()
monkeypatch.setenv("RAGFLOW_API_URL", "http://api")
monkeypatch.delenv("RAGFLOW_API_KEY", raising=False)
with pytest.raises(ValueError):
RAGFlowProvider()
@patch("src.rag.ragflow.requests.post")
def test_query_relevant_documents_success(mock_post, monkeypatch):
monkeypatch.setenv("RAGFLOW_API_URL", "http://api")
monkeypatch.setenv("RAGFLOW_API_KEY", "key")
provider = RAGFlowProvider()
resource = DummyResource("rag://dataset/123#doc456")
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
"data": {
"doc_aggs": [{"doc_id": "doc456", "doc_name": "Doc Title"}],
"chunks": [
{"document_id": "doc456", "content": "chunk text", "similarity": 0.9}
],
}
}
mock_post.return_value = mock_response
docs = provider.query_relevant_documents("query", [resource])
assert len(docs) == 1
assert docs[0].id == "doc456"
assert docs[0].title == "Doc Title"
assert len(docs[0].chunks) == 1
assert docs[0].chunks[0].content == "chunk text"
assert docs[0].chunks[0].similarity == 0.9
@patch("src.rag.ragflow.requests.post")
def test_query_relevant_documents_error(mock_post, monkeypatch):
monkeypatch.setenv("RAGFLOW_API_URL", "http://api")
monkeypatch.setenv("RAGFLOW_API_KEY", "key")
provider = RAGFlowProvider()
mock_response = MagicMock()
mock_response.status_code = 400
mock_response.text = "error"
mock_post.return_value = mock_response
with pytest.raises(Exception):
provider.query_relevant_documents("query", [])
@patch("src.rag.ragflow.requests.get")
def test_list_resources_success(mock_get, monkeypatch):
monkeypatch.setenv("RAGFLOW_API_URL", "http://api")
monkeypatch.setenv("RAGFLOW_API_KEY", "key")
provider = RAGFlowProvider()
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
"data": [
{"id": "123", "name": "Dataset1", "description": "desc1"},
{"id": "456", "name": "Dataset2", "description": "desc2"},
]
}
mock_get.return_value = mock_response
resources = provider.list_resources()
assert len(resources) == 2
assert resources[0].uri == "rag://dataset/123"
assert resources[0].title == "Dataset1"
assert resources[0].description == "desc1"
assert resources[1].uri == "rag://dataset/456"
assert resources[1].title == "Dataset2"
assert resources[1].description == "desc2"
@patch("src.rag.ragflow.requests.get")
def test_list_resources_error(mock_get, monkeypatch):
monkeypatch.setenv("RAGFLOW_API_URL", "http://api")
monkeypatch.setenv("RAGFLOW_API_KEY", "key")
provider = RAGFlowProvider()
mock_response = MagicMock()
mock_response.status_code = 500
mock_response.text = "fail"
mock_get.return_value = mock_response
with pytest.raises(Exception):
provider.list_resources()
+72
View File
@@ -0,0 +1,72 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from src.rag.retriever import Chunk, Document, Resource, Retriever
def test_chunk_init():
chunk = Chunk(content="test content", similarity=0.9)
assert chunk.content == "test content"
assert chunk.similarity == 0.9
def test_document_init_and_to_dict():
chunk1 = Chunk(content="chunk1", similarity=0.8)
chunk2 = Chunk(content="chunk2", similarity=0.7)
doc = Document(
id="doc1", url="http://example.com", title="Title", chunks=[chunk1, chunk2]
)
assert doc.id == "doc1"
assert doc.url == "http://example.com"
assert doc.title == "Title"
assert doc.chunks == [chunk1, chunk2]
d = doc.to_dict()
assert d["id"] == "doc1"
assert d["content"] == "chunk1\n\nchunk2"
assert d["url"] == "http://example.com"
assert d["title"] == "Title"
def test_document_to_dict_optional_fields():
chunk = Chunk(content="only chunk", similarity=1.0)
doc = Document(id="doc2", chunks=[chunk])
d = doc.to_dict()
assert d["id"] == "doc2"
assert d["content"] == "only chunk"
assert "url" not in d
assert "title" not in d
def test_resource_model():
resource = Resource(uri="uri1", title="Resource Title")
assert resource.uri == "uri1"
assert resource.title == "Resource Title"
assert resource.description == ""
def test_resource_model_with_description():
resource = Resource(uri="uri2", title="Resource2", description="desc")
assert resource.description == "desc"
def test_retriever_abstract_methods():
class DummyRetriever(Retriever):
def list_resources(self, query=None):
return [Resource(uri="uri", title="title")]
def query_relevant_documents(self, query, resources=[]):
return [Document(id="id", chunks=[])]
retriever = DummyRetriever()
resources = retriever.list_resources()
assert isinstance(resources, list)
assert isinstance(resources[0], Resource)
docs = retriever.query_relevant_documents("query", resources)
assert isinstance(docs, list)
assert isinstance(docs[0], Document)
def test_retriever_cannot_instantiate():
with pytest.raises(TypeError):
Retriever()
@@ -0,0 +1,503 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import os
import pytest
import json
from unittest.mock import patch, MagicMock
from src.rag.vikingdb_knowledge_base import VikingDBKnowledgeBaseProvider, parse_uri
# Dummy classes to mock dependencies
class MockResource:
def __init__(self, uri, title="", description=""):
self.uri = uri
self.title = title
self.description = description
class MockChunk:
def __init__(self, content, similarity):
self.content = content
self.similarity = similarity
class MockDocument:
def __init__(self, id, title, chunks=None):
self.id = id
self.title = title
self.chunks = chunks or []
# Patch the imports to use mock classes
@pytest.fixture(autouse=True)
def patch_imports():
with (
patch("src.rag.vikingdb_knowledge_base.Resource", MockResource),
patch("src.rag.vikingdb_knowledge_base.Chunk", MockChunk),
patch("src.rag.vikingdb_knowledge_base.Document", MockDocument),
):
yield
@pytest.fixture
def env_vars():
"""Fixture to set up environment variables"""
with patch.dict(
os.environ,
{
"VIKINGDB_KNOWLEDGE_BASE_API_URL": "api-test.example.com",
"VIKINGDB_KNOWLEDGE_BASE_API_AK": "test_ak",
"VIKINGDB_KNOWLEDGE_BASE_API_SK": "test_sk",
"VIKINGDB_KNOWLEDGE_BASE_RETRIEVAL_SIZE": "10",
},
):
yield
class TestParseUri:
def test_parse_uri_valid_with_fragment(self):
"""Test parsing valid URI with fragment"""
uri = "rag://dataset/123#doc456"
resource_id, document_id = parse_uri(uri)
assert resource_id == "123"
assert document_id == "doc456"
def test_parse_uri_valid_without_fragment(self):
"""Test parsing valid URI without fragment"""
uri = "rag://dataset/123"
resource_id, document_id = parse_uri(uri)
assert resource_id == "123"
assert document_id == ""
def test_parse_uri_invalid_scheme(self):
"""Test parsing URI with invalid scheme"""
with pytest.raises(ValueError, match="Invalid URI"):
parse_uri("http://dataset/123#abc")
def test_parse_uri_malformed(self):
"""Test parsing malformed URI"""
with pytest.raises(ValueError, match="Invalid URI"):
parse_uri("invalid_uri")
class TestVikingDBKnowledgeBaseProviderInit:
def test_init_success_with_all_env_vars(self, env_vars):
"""Test successful initialization with all environment variables"""
provider = VikingDBKnowledgeBaseProvider()
assert provider.api_url == "api-test.example.com"
assert provider.api_ak == "test_ak"
assert provider.api_sk == "test_sk"
assert provider.retrieval_size == 10
def test_init_success_without_retrieval_size(self):
"""Test initialization without VIKINGDB_KNOWLEDGE_BASE_RETRIEVAL_SIZE (should use default)"""
with patch.dict(
os.environ,
{
"VIKINGDB_KNOWLEDGE_BASE_API_URL": "api-test.example.com",
"VIKINGDB_KNOWLEDGE_BASE_API_AK": "test_ak",
"VIKINGDB_KNOWLEDGE_BASE_API_SK": "test_sk",
},
clear=True,
):
provider = VikingDBKnowledgeBaseProvider()
assert provider.retrieval_size == 10
def test_init_custom_retrieval_size(self):
"""Test initialization with custom retrieval size"""
with patch.dict(
os.environ,
{
"VIKINGDB_KNOWLEDGE_BASE_API_URL": "api-test.example.com",
"VIKINGDB_KNOWLEDGE_BASE_API_AK": "test_ak",
"VIKINGDB_KNOWLEDGE_BASE_API_SK": "test_sk",
"VIKINGDB_KNOWLEDGE_BASE_RETRIEVAL_SIZE": "5",
},
):
provider = VikingDBKnowledgeBaseProvider()
assert provider.retrieval_size == 5
def test_init_missing_api_url(self):
"""Test initialization fails when API URL is missing"""
with patch.dict(
os.environ,
{
"VIKINGDB_KNOWLEDGE_BASE_API_AK": "test_ak",
"VIKINGDB_KNOWLEDGE_BASE_API_SK": "test_sk",
},
clear=True,
):
with pytest.raises(
ValueError, match="VIKINGDB_KNOWLEDGE_BASE_API_URL is not set"
):
VikingDBKnowledgeBaseProvider()
def test_init_missing_api_ak(self):
"""Test initialization fails when API AK is missing"""
with patch.dict(
os.environ,
{
"VIKINGDB_KNOWLEDGE_BASE_API_URL": "api-test.example.com",
"VIKINGDB_KNOWLEDGE_BASE_API_SK": "test_sk",
},
clear=True,
):
with pytest.raises(
ValueError, match="VIKINGDB_KNOWLEDGE_BASE_API_AK is not set"
):
VikingDBKnowledgeBaseProvider()
def test_init_missing_api_sk(self):
"""Test initialization fails when API SK is missing"""
with patch.dict(
os.environ,
{
"VIKINGDB_KNOWLEDGE_BASE_API_URL": "api-test.example.com",
"VIKINGDB_KNOWLEDGE_BASE_API_AK": "test_ak",
},
clear=True,
):
with pytest.raises(
ValueError, match="VIKINGDB_KNOWLEDGE_BASE_API_SK is not set"
):
VikingDBKnowledgeBaseProvider()
class TestVikingDBKnowledgeBaseProviderPrepareRequest:
@pytest.fixture
def provider(self, env_vars):
return VikingDBKnowledgeBaseProvider()
def test_prepare_request_basic(self, provider):
"""Test basic request preparation"""
with (
patch("src.rag.vikingdb_knowledge_base.Request") as mock_request,
patch("src.rag.vikingdb_knowledge_base.Credentials") as _mock_credentials,
patch("src.rag.vikingdb_knowledge_base.SignerV4.sign") as _mock_sign,
):
mock_req_instance = MagicMock()
mock_request.return_value = mock_req_instance
result = provider.prepare_request("POST", "/test/path")
assert result == mock_req_instance
mock_req_instance.set_shema.assert_called_once_with("https")
mock_req_instance.set_method.assert_called_once_with("POST")
mock_req_instance.set_path.assert_called_once_with("/test/path")
def test_prepare_request_with_params(self, provider):
"""Test request preparation with parameters"""
with (
patch("src.rag.vikingdb_knowledge_base.Request") as mock_request,
patch("src.rag.vikingdb_knowledge_base.Credentials"),
patch("src.rag.vikingdb_knowledge_base.SignerV4.sign"),
):
mock_req_instance = MagicMock()
mock_request.return_value = mock_req_instance
params = {"key": "value", "number": 123, "boolean": True}
provider.prepare_request("GET", "/test", params=params)
expected_params = {"key": "value", "number": "123", "boolean": "True"}
mock_req_instance.set_query.assert_called_once_with(expected_params)
def test_prepare_request_with_data(self, provider):
"""Test request preparation with data"""
with (
patch("src.rag.vikingdb_knowledge_base.Request") as mock_request,
patch("src.rag.vikingdb_knowledge_base.Credentials"),
patch("src.rag.vikingdb_knowledge_base.SignerV4.sign"),
):
mock_req_instance = MagicMock()
mock_request.return_value = mock_req_instance
data = {"test": "data"}
provider.prepare_request("POST", "/test", data=data)
mock_req_instance.set_body.assert_called_once_with(json.dumps(data))
class TestVikingDBKnowledgeBaseProviderQueryRelevantDocuments:
@pytest.fixture
def provider(self, env_vars):
return VikingDBKnowledgeBaseProvider()
def test_query_relevant_documents_empty_resources(self, provider):
"""Test querying with empty resources list"""
result = provider.query_relevant_documents("test query", [])
assert result == []
@patch("src.rag.vikingdb_knowledge_base.requests.request")
def test_query_relevant_documents_success(self, mock_request, provider):
"""Test successful document query"""
# Mock response
mock_response = MagicMock()
mock_response.text = json.dumps(
{
"code": 0,
"data": {
"result_list": [
{
"doc_info": {
"doc_id": "doc123",
"doc_name": "Test Document",
},
"content": "Test content",
"score": 0.95,
}
]
},
}
)
mock_request.return_value = mock_response
# Mock prepare_request
with patch.object(provider, "prepare_request") as mock_prepare:
mock_req = MagicMock()
mock_req.method = "POST"
mock_req.path = "/api/knowledge/collection/search_knowledge"
mock_req.headers = {}
mock_req.body = "{}"
mock_prepare.return_value = mock_req
resources = [MockResource("rag://dataset/123")]
result = provider.query_relevant_documents("test query", resources)
assert len(result) == 1
assert result[0].id == "doc123"
assert result[0].title == "Test Document"
assert len(result[0].chunks) == 1
assert result[0].chunks[0].content == "Test content"
assert result[0].chunks[0].similarity == 0.95
@patch("src.rag.vikingdb_knowledge_base.requests.request")
def test_query_relevant_documents_with_document_filter(
self, mock_request, provider
):
"""Test document query with document ID filter"""
mock_response = MagicMock()
mock_response.text = json.dumps({"code": 0, "data": {"result_list": []}})
mock_request.return_value = mock_response
with patch.object(provider, "prepare_request") as mock_prepare:
mock_req = MagicMock()
mock_prepare.return_value = mock_req
resources = [MockResource("rag://dataset/123#doc456")]
provider.query_relevant_documents("test query", resources)
# Verify that query_param with doc_filter was included in the request
call_args = mock_prepare.call_args
request_data = call_args[1]["data"]
assert "query_param" in request_data
assert "doc_filter" in request_data["query_param"]
doc_filter = request_data["query_param"]["doc_filter"]
assert doc_filter["op"] == "must"
assert doc_filter["field"] == "doc_id"
assert doc_filter["conds"] == ["doc456"]
@patch("src.rag.vikingdb_knowledge_base.requests.request")
def test_query_relevant_documents_api_error(self, mock_request, provider):
"""Test handling of API error response"""
mock_response = MagicMock()
mock_response.text = json.dumps({"code": 1, "message": "API Error"})
mock_request.return_value = mock_response
with patch.object(provider, "prepare_request"):
resources = [MockResource("rag://dataset/123")]
with pytest.raises(
ValueError, match="Failed to query documents from resource: API Error"
):
provider.query_relevant_documents("test query", resources)
@patch("src.rag.vikingdb_knowledge_base.requests.request")
def test_query_relevant_documents_json_decode_error(self, mock_request, provider):
"""Test handling of JSON decode error"""
mock_response = MagicMock()
mock_response.text = "invalid json"
mock_request.return_value = mock_response
with patch.object(provider, "prepare_request"):
resources = [MockResource("rag://dataset/123")]
with pytest.raises(ValueError, match="Failed to parse JSON response"):
provider.query_relevant_documents("test query", resources)
@patch("src.rag.vikingdb_knowledge_base.requests.request")
def test_query_relevant_documents_multiple_resources(self, mock_request, provider):
"""Test querying multiple resources and merging results"""
# Mock responses for different resources
responses = [
json.dumps(
{
"code": 0,
"data": {
"result_list": [
{
"doc_info": {
"doc_id": "doc1",
"doc_name": "Document 1",
},
"content": "Content 1",
"score": 0.9,
}
]
},
}
),
json.dumps(
{
"code": 0,
"data": {
"result_list": [
{
"doc_info": {
"doc_id": "doc1",
"doc_name": "Document 1",
},
"content": "Content 2",
"score": 0.8,
},
{
"doc_info": {
"doc_id": "doc2",
"doc_name": "Document 2",
},
"content": "Content 3",
"score": 0.7,
},
]
},
}
),
]
mock_request.side_effect = [MagicMock(text=resp) for resp in responses]
with patch.object(provider, "prepare_request"):
resources = [
MockResource("rag://dataset/123"),
MockResource("rag://dataset/456"),
]
result = provider.query_relevant_documents("test query", resources)
# Should have 2 documents: doc1 (with 2 chunks) and doc2 (with 1 chunk)
assert len(result) == 2
doc1 = next(doc for doc in result if doc.id == "doc1")
doc2 = next(doc for doc in result if doc.id == "doc2")
assert len(doc1.chunks) == 2
assert len(doc2.chunks) == 1
class TestVikingDBKnowledgeBaseProviderListResources:
@pytest.fixture
def provider(self, env_vars):
return VikingDBKnowledgeBaseProvider()
@patch("src.rag.vikingdb_knowledge_base.requests.request")
def test_list_resources_success(self, mock_request, provider):
"""Test successful resource listing"""
mock_response = MagicMock()
mock_response.text = json.dumps(
{
"code": 0,
"data": {
"collection_list": [
{
"resource_id": "123",
"collection_name": "Dataset 1",
"description": "Description 1",
},
{
"resource_id": "456",
"collection_name": "Dataset 2",
"description": "Description 2",
},
]
},
}
)
mock_request.return_value = mock_response
with patch.object(provider, "prepare_request") as mock_prepare:
mock_req = MagicMock()
mock_prepare.return_value = mock_req
result = provider.list_resources()
assert len(result) == 2
assert result[0].uri == "rag://dataset/123"
assert result[0].title == "Dataset 1"
assert result[0].description == "Description 1"
assert result[1].uri == "rag://dataset/456"
assert result[1].title == "Dataset 2"
assert result[1].description == "Description 2"
@patch("src.rag.vikingdb_knowledge_base.requests.request")
def test_list_resources_with_query_filter(self, mock_request, provider):
"""Test resource listing with query filter"""
mock_response = MagicMock()
mock_response.text = json.dumps(
{
"code": 0,
"data": {
"collection_list": [
{
"resource_id": "123",
"collection_name": "Test Dataset",
"description": "Description",
},
{
"resource_id": "456",
"collection_name": "Other Dataset",
"description": "Description",
},
]
},
}
)
mock_request.return_value = mock_response
with patch.object(provider, "prepare_request"):
result = provider.list_resources("test")
# Should only return the dataset with "test" in the name
assert len(result) == 1
assert result[0].title == "Test Dataset"
@patch("src.rag.vikingdb_knowledge_base.requests.request")
def test_list_resources_api_error(self, mock_request, provider):
"""Test handling of API error in list_resources"""
mock_response = MagicMock()
mock_response.text = json.dumps({"code": 1, "message": "API Error"})
mock_request.return_value = mock_response
with patch.object(provider, "prepare_request"):
with pytest.raises(Exception, match="Failed to list resources: API Error"):
provider.list_resources()
@patch("src.rag.vikingdb_knowledge_base.requests.request")
def test_list_resources_json_decode_error(self, mock_request, provider):
"""Test handling of JSON decode error in list_resources"""
mock_response = MagicMock()
mock_response.text = "invalid json"
mock_request.return_value = mock_response
with patch.object(provider, "prepare_request"):
with pytest.raises(ValueError, match="Failed to parse JSON response"):
provider.list_resources()
@patch("src.rag.vikingdb_knowledge_base.requests.request")
def test_list_resources_empty_response(self, mock_request, provider):
"""Test handling of empty response"""
mock_response = MagicMock()
mock_response.text = json.dumps({"code": 0, "data": {"collection_list": []}})
mock_request.return_value = mock_response
with patch.object(provider, "prepare_request"):
result = provider.list_resources()
assert result == []
+841
View File
@@ -0,0 +1,841 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import base64
import os
from unittest.mock import MagicMock, patch, mock_open
import pytest
from fastapi.testclient import TestClient
from fastapi import HTTPException
from src.server.app import app, _make_event, _astream_workflow_generator
from src.config.report_style import ReportStyle
from langgraph.types import Command
from langchain_core.messages import ToolMessage
from langchain_core.messages import AIMessageChunk
@pytest.fixture
def client():
return TestClient(app)
class TestMakeEvent:
def test_make_event_with_content(self):
event_type = "message_chunk"
data = {"content": "Hello", "role": "assistant"}
result = _make_event(event_type, data)
expected = (
'event: message_chunk\ndata: {"content": "Hello", "role": "assistant"}\n\n'
)
assert result == expected
def test_make_event_with_empty_content(self):
event_type = "message_chunk"
data = {"content": "", "role": "assistant"}
result = _make_event(event_type, data)
expected = 'event: message_chunk\ndata: {"role": "assistant"}\n\n'
assert result == expected
def test_make_event_without_content(self):
event_type = "tool_calls"
data = {"role": "assistant", "tool_calls": []}
result = _make_event(event_type, data)
expected = (
'event: tool_calls\ndata: {"role": "assistant", "tool_calls": []}\n\n'
)
assert result == expected
class TestTTSEndpoint:
@patch.dict(
os.environ,
{
"VOLCENGINE_TTS_APPID": "test_app_id",
"VOLCENGINE_TTS_ACCESS_TOKEN": "test_token",
"VOLCENGINE_TTS_CLUSTER": "test_cluster",
"VOLCENGINE_TTS_VOICE_TYPE": "test_voice",
},
)
@patch("src.server.app.VolcengineTTS")
def test_tts_success(self, mock_tts_class, client):
mock_tts_instance = MagicMock()
mock_tts_class.return_value = mock_tts_instance
# Mock successful TTS response
audio_data_b64 = base64.b64encode(b"fake_audio_data").decode()
mock_tts_instance.text_to_speech.return_value = {
"success": True,
"audio_data": audio_data_b64,
}
request_data = {
"text": "Hello world",
"encoding": "mp3",
"speed_ratio": 1.0,
"volume_ratio": 1.0,
"pitch_ratio": 1.0,
"text_type": "plain",
"with_frontend": True,
"frontend_type": "unitTson",
}
response = client.post("/api/tts", json=request_data)
assert response.status_code == 200
assert response.headers["content-type"] == "audio/mp3"
assert b"fake_audio_data" in response.content
@patch.dict(os.environ, {}, clear=True)
def test_tts_missing_app_id(self, client):
request_data = {"text": "Hello world", "encoding": "mp3"}
response = client.post("/api/tts", json=request_data)
assert response.status_code == 400
assert "VOLCENGINE_TTS_APPID is not set" in response.json()["detail"]
@patch.dict(
os.environ,
{"VOLCENGINE_TTS_APPID": "test_app_id", "VOLCENGINE_TTS_ACCESS_TOKEN": ""},
)
def test_tts_missing_access_token(self, client):
request_data = {"text": "Hello world", "encoding": "mp3"}
response = client.post("/api/tts", json=request_data)
assert response.status_code == 400
assert "VOLCENGINE_TTS_ACCESS_TOKEN is not set" in response.json()["detail"]
@patch.dict(
os.environ,
{
"VOLCENGINE_TTS_APPID": "test_app_id",
"VOLCENGINE_TTS_ACCESS_TOKEN": "test_token",
},
)
@patch("src.server.app.VolcengineTTS")
def test_tts_api_error(self, mock_tts_class, client):
mock_tts_instance = MagicMock()
mock_tts_class.return_value = mock_tts_instance
# Mock TTS error response
mock_tts_instance.text_to_speech.return_value = {
"success": False,
"error": "TTS API error",
}
request_data = {"text": "Hello world", "encoding": "mp3"}
response = client.post("/api/tts", json=request_data)
assert response.status_code == 500
assert "Internal Server Error" in response.json()["detail"]
@pytest.mark.skip(reason="TTS server exception is catched")
@patch("src.server.app.VolcengineTTS")
def test_tts_api_exception(self, mock_tts_class, client):
mock_tts_instance = MagicMock()
mock_tts_class.return_value = mock_tts_instance
# Mock TTS error response
mock_tts_instance.side_effect = Exception("TTS API error")
request_data = {"text": "Hello world", "encoding": "mp3"}
response = client.post("/api/tts", json=request_data)
assert response.status_code == 500
assert "Internal Server Error" in response.json()["detail"]
class TestPodcastEndpoint:
@patch("src.server.app.build_podcast_graph")
def test_generate_podcast_success(self, mock_build_graph, client):
mock_workflow = MagicMock()
mock_build_graph.return_value = mock_workflow
mock_workflow.invoke.return_value = {"output": b"fake_audio_data"}
request_data = {"content": "Test content for podcast"}
response = client.post("/api/podcast/generate", json=request_data)
assert response.status_code == 200
assert response.headers["content-type"] == "audio/mp3"
assert response.content == b"fake_audio_data"
@patch("src.server.app.build_podcast_graph")
def test_generate_podcast_error(self, mock_build_graph, client):
mock_build_graph.side_effect = Exception("Podcast generation failed")
request_data = {"content": "Test content"}
response = client.post("/api/podcast/generate", json=request_data)
assert response.status_code == 500
assert response.json()["detail"] == "Internal Server Error"
class TestPPTEndpoint:
@patch("src.server.app.build_ppt_graph")
@patch("builtins.open", new_callable=mock_open, read_data=b"fake_ppt_data")
def test_generate_ppt_success(self, mock_file, mock_build_graph, client):
mock_workflow = MagicMock()
mock_build_graph.return_value = mock_workflow
mock_workflow.invoke.return_value = {
"generated_file_path": "/fake/path/test.pptx"
}
request_data = {"content": "Test content for PPT"}
response = client.post("/api/ppt/generate", json=request_data)
assert response.status_code == 200
assert (
"application/vnd.openxmlformats-officedocument.presentationml.presentation"
in response.headers["content-type"]
)
assert response.content == b"fake_ppt_data"
@patch("src.server.app.build_ppt_graph")
def test_generate_ppt_error(self, mock_build_graph, client):
mock_build_graph.side_effect = Exception("PPT generation failed")
request_data = {"content": "Test content"}
response = client.post("/api/ppt/generate", json=request_data)
assert response.status_code == 500
assert response.json()["detail"] == "Internal Server Error"
class TestEnhancePromptEndpoint:
@patch("src.server.app.build_prompt_enhancer_graph")
def test_enhance_prompt_success(self, mock_build_graph, client):
mock_workflow = MagicMock()
mock_build_graph.return_value = mock_workflow
mock_workflow.invoke.return_value = {"output": "Enhanced prompt"}
request_data = {
"prompt": "Original prompt",
"context": "Some context",
"report_style": "academic",
}
response = client.post("/api/prompt/enhance", json=request_data)
assert response.status_code == 200
assert response.json()["result"] == "Enhanced prompt"
@patch("src.server.app.build_prompt_enhancer_graph")
def test_enhance_prompt_with_different_styles(self, mock_build_graph, client):
mock_workflow = MagicMock()
mock_build_graph.return_value = mock_workflow
mock_workflow.invoke.return_value = {"output": "Enhanced prompt"}
styles = [
"ACADEMIC",
"popular_science",
"NEWS",
"social_media",
"invalid_style",
]
for style in styles:
request_data = {"prompt": "Test prompt", "report_style": style}
response = client.post("/api/prompt/enhance", json=request_data)
assert response.status_code == 200
@patch("src.server.app.build_prompt_enhancer_graph")
def test_enhance_prompt_error(self, mock_build_graph, client):
mock_build_graph.side_effect = Exception("Enhancement failed")
request_data = {"prompt": "Test prompt"}
response = client.post("/api/prompt/enhance", json=request_data)
assert response.status_code == 500
assert response.json()["detail"] == "Internal Server Error"
class TestMCPEndpoint:
@patch("src.server.app.load_mcp_tools")
@patch.dict(
os.environ,
{"ENABLE_MCP_SERVER_CONFIGURATION": "true"},
)
def test_mcp_server_metadata_success(self, mock_load_tools, client):
mock_load_tools.return_value = [
{"name": "test_tool", "description": "Test tool"}
]
request_data = {
"transport": "stdio",
"command": "test_command",
"args": ["arg1", "arg2"],
"env": {"ENV_VAR": "value"},
}
response = client.post("/api/mcp/server/metadata", json=request_data)
assert response.status_code == 200
response_data = response.json()
assert response_data["transport"] == "stdio"
assert response_data["command"] == "test_command"
assert len(response_data["tools"]) == 1
@patch("src.server.app.load_mcp_tools")
@patch.dict(
os.environ,
{"ENABLE_MCP_SERVER_CONFIGURATION": "true"},
)
def test_mcp_server_metadata_with_custom_timeout(self, mock_load_tools, client):
mock_load_tools.return_value = []
request_data = {
"transport": "stdio",
"command": "test_command",
"timeout_seconds": 600,
}
response = client.post("/api/mcp/server/metadata", json=request_data)
assert response.status_code == 200
mock_load_tools.assert_called_once()
@patch("src.server.app.load_mcp_tools")
@patch.dict(
os.environ,
{"ENABLE_MCP_SERVER_CONFIGURATION": "true"},
)
def test_mcp_server_metadata_with_exception(self, mock_load_tools, client):
mock_load_tools.side_effect = HTTPException(
status_code=400, detail="MCP Server Error"
)
request_data = {
"transport": "stdio",
"command": "test_command",
"args": ["arg1", "arg2"],
"env": {"ENV_VAR": "value"},
}
response = client.post("/api/mcp/server/metadata", json=request_data)
assert response.status_code == 500
assert response.json()["detail"] == "Internal Server Error"
@patch("src.server.app.load_mcp_tools")
@patch.dict(
os.environ,
{"ENABLE_MCP_SERVER_CONFIGURATION": ""},
)
def test_mcp_server_metadata_without_enable_configuration(
self, mock_load_tools, client
):
request_data = {
"transport": "stdio",
"command": "test_command",
"args": ["arg1", "arg2"],
"env": {"ENV_VAR": "value"},
}
response = client.post("/api/mcp/server/metadata", json=request_data)
assert response.status_code == 403
assert (
response.json()["detail"]
== "MCP server configuration is disabled. Set ENABLE_MCP_SERVER_CONFIGURATION=true to enable MCP features."
)
class TestRAGEndpoints:
@patch("src.server.app.SELECTED_RAG_PROVIDER", "test_provider")
def test_rag_config(self, client):
response = client.get("/api/rag/config")
assert response.status_code == 200
assert response.json()["provider"] == "test_provider"
@patch("src.server.app.build_retriever")
def test_rag_resources_with_retriever(self, mock_build_retriever, client):
mock_retriever = MagicMock()
mock_retriever.list_resources.return_value = [
{
"uri": "test_uri",
"title": "Test Resource",
"description": "Test Description",
}
]
mock_build_retriever.return_value = mock_retriever
response = client.get("/api/rag/resources?query=test")
assert response.status_code == 200
assert len(response.json()["resources"]) == 1
@patch("src.server.app.build_retriever")
def test_rag_resources_without_retriever(self, mock_build_retriever, client):
mock_build_retriever.return_value = None
response = client.get("/api/rag/resources")
assert response.status_code == 200
assert response.json()["resources"] == []
class TestChatStreamEndpoint:
@patch("src.server.app.graph")
def test_chat_stream_with_default_thread_id(self, mock_graph, client):
# Mock the async stream
async def mock_astream(*args, **kwargs):
yield ("agent1", "step1", {"test": "data"})
mock_graph.astream = mock_astream
request_data = {
"thread_id": "__default__",
"messages": [{"role": "user", "content": "Hello"}],
"resources": [],
"max_plan_iterations": 3,
"max_step_num": 10,
"max_search_results": 5,
"auto_accepted_plan": True,
"interrupt_feedback": "",
"mcp_settings": {},
"enable_background_investigation": False,
"report_style": "academic",
}
response = client.post("/api/chat/stream", json=request_data)
assert response.status_code == 200
assert response.headers["content-type"] == "text/event-stream; charset=utf-8"
@patch("src.server.app.graph")
def test_chat_stream_with_mcp_settings(self, mock_graph, client):
# Mock the async stream
async def mock_astream(*args, **kwargs):
yield ("agent1", "step1", {"test": "data"})
mock_graph.astream = mock_astream
request_data = {
"thread_id": "__default__",
"messages": [{"role": "user", "content": "Hello"}],
"resources": [],
"max_plan_iterations": 3,
"max_step_num": 10,
"max_search_results": 5,
"auto_accepted_plan": True,
"interrupt_feedback": "",
"mcp_settings": {
"servers": {
"mcp-github-trending": {
"transport": "stdio",
"command": "uvx",
"args": ["mcp-github-trending"],
"env": {"MCP_SERVER_ID": "mcp-github-trending"},
"enabled_tools": ["get_github_trending_repositories"],
"add_to_agents": ["researcher"],
}
}
},
"enable_background_investigation": False,
"report_style": "academic",
}
response = client.post("/api/chat/stream", json=request_data)
assert response.status_code == 403
assert (
response.json()["detail"]
== "MCP server configuration is disabled. Set ENABLE_MCP_SERVER_CONFIGURATION=true to enable MCP features."
)
@patch("src.server.app.graph")
@patch.dict(
os.environ,
{"ENABLE_MCP_SERVER_CONFIGURATION": "true"},
)
def test_chat_stream_with_mcp_settings_enabled(self, mock_graph, client):
# Mock the async stream
async def mock_astream(*args, **kwargs):
yield ("agent1", "step1", {"test": "data"})
mock_graph.astream = mock_astream
request_data = {
"thread_id": "__default__",
"messages": [{"role": "user", "content": "Hello"}],
"resources": [],
"max_plan_iterations": 3,
"max_step_num": 10,
"max_search_results": 5,
"auto_accepted_plan": True,
"interrupt_feedback": "",
"mcp_settings": {
"servers": {
"mcp-github-trending": {
"transport": "stdio",
"command": "uvx",
"args": ["mcp-github-trending"],
"env": {"MCP_SERVER_ID": "mcp-github-trending"},
"enabled_tools": ["get_github_trending_repositories"],
"add_to_agents": ["researcher"],
}
}
},
"enable_background_investigation": False,
"report_style": "academic",
}
response = client.post("/api/chat/stream", json=request_data)
assert response.status_code == 200
assert response.headers["content-type"] == "text/event-stream; charset=utf-8"
class TestAstreamWorkflowGenerator:
@pytest.mark.asyncio
@patch("src.server.app.graph")
async def test_astream_workflow_generator_basic_flow(self, mock_graph):
# Mock AI message chunk
mock_message = AIMessageChunk(content="Hello world")
mock_message.id = "msg_123"
mock_message.response_metadata = {}
mock_message.tool_calls = []
mock_message.tool_call_chunks = []
# Mock the async stream - yield messages in the correct format
async def mock_astream(*args, **kwargs):
# Yield a tuple (message, metadata) instead of just [message]
yield ("agent1:subagent", "messages", (mock_message, {}))
mock_graph.astream = mock_astream
messages = [{"role": "user", "content": "Hello"}]
thread_id = "test_thread"
resources = []
generator = _astream_workflow_generator(
messages=messages,
thread_id=thread_id,
resources=resources,
max_plan_iterations=3,
max_step_num=10,
max_search_results=5,
auto_accepted_plan=True,
interrupt_feedback="",
mcp_settings={},
enable_background_investigation=False,
report_style=ReportStyle.ACADEMIC,
enable_deep_thinking=False,
)
events = []
async for event in generator:
events.append(event)
assert len(events) == 1
assert "event: message_chunk" in events[0]
assert "Hello world" in events[0]
# Check for the actual agent name that appears in the output
assert '"agent": "a"' in events[0]
@pytest.mark.asyncio
@patch("src.server.app.graph")
async def test_astream_workflow_generator_with_interrupt_feedback(self, mock_graph):
# Mock the async stream
async def mock_astream(*args, **kwargs):
# Verify that Command is passed as input when interrupt_feedback is provided
assert isinstance(args[0], Command)
assert "[edit_plan] Hello" in args[0].resume
yield ("agent1", "step1", {"test": "data"})
mock_graph.astream = mock_astream
messages = [{"role": "user", "content": "Hello"}]
generator = _astream_workflow_generator(
messages=messages,
thread_id="test_thread",
resources=[],
max_plan_iterations=3,
max_step_num=10,
max_search_results=5,
auto_accepted_plan=False,
interrupt_feedback="edit_plan",
mcp_settings={},
enable_background_investigation=False,
report_style=ReportStyle.ACADEMIC,
enable_deep_thinking=False,
)
events = []
async for event in generator:
events.append(event)
@pytest.mark.asyncio
@patch("src.server.app.graph")
async def test_astream_workflow_generator_interrupt_event(self, mock_graph):
# Mock interrupt data
mock_interrupt = MagicMock()
mock_interrupt.ns = ["interrupt_id"]
mock_interrupt.value = "Plan requires approval"
interrupt_data = {"__interrupt__": [mock_interrupt]}
async def mock_astream(*args, **kwargs):
yield ("agent1", "step1", interrupt_data)
mock_graph.astream = mock_astream
generator = _astream_workflow_generator(
messages=[],
thread_id="test_thread",
resources=[],
max_plan_iterations=3,
max_step_num=10,
max_search_results=5,
auto_accepted_plan=True,
interrupt_feedback="",
mcp_settings={},
enable_background_investigation=False,
report_style=ReportStyle.ACADEMIC,
enable_deep_thinking=False,
)
events = []
async for event in generator:
events.append(event)
assert len(events) == 1
assert "event: interrupt" in events[0]
assert "Plan requires approval" in events[0]
assert "interrupt_id" in events[0]
@pytest.mark.asyncio
@patch("src.server.app.graph")
async def test_astream_workflow_generator_tool_message(self, mock_graph):
# Mock tool message
mock_tool_message = ToolMessage(content="Tool result", tool_call_id="tool_123")
mock_tool_message.id = "msg_456"
async def mock_astream(*args, **kwargs):
yield ("agent1:subagent", "step1", (mock_tool_message, {}))
mock_graph.astream = mock_astream
generator = _astream_workflow_generator(
messages=[],
thread_id="test_thread",
resources=[],
max_plan_iterations=3,
max_step_num=10,
max_search_results=5,
auto_accepted_plan=True,
interrupt_feedback="",
mcp_settings={},
enable_background_investigation=False,
report_style=ReportStyle.ACADEMIC,
enable_deep_thinking=False,
)
events = []
async for event in generator:
events.append(event)
assert len(events) == 1
assert "event: tool_call_result" in events[0]
assert "Tool result" in events[0]
assert "tool_123" in events[0]
@pytest.mark.asyncio
@patch("src.server.app.graph")
async def test_astream_workflow_generator_ai_message_with_tool_calls(
self, mock_graph
):
# Mock AI message with tool calls
mock_ai_message = AIMessageChunk(content="Making tool call")
mock_ai_message.id = "msg_789"
mock_ai_message.response_metadata = {"finish_reason": "tool_calls"}
mock_ai_message.tool_calls = [{"name": "search", "args": {"query": "test"}}]
mock_ai_message.tool_call_chunks = [{"name": "search"}]
async def mock_astream(*args, **kwargs):
yield ("agent1:subagent", "step1", (mock_ai_message, {}))
mock_graph.astream = mock_astream
generator = _astream_workflow_generator(
messages=[],
thread_id="test_thread",
resources=[],
max_plan_iterations=3,
max_step_num=10,
max_search_results=5,
auto_accepted_plan=True,
interrupt_feedback="",
mcp_settings={},
enable_background_investigation=False,
report_style=ReportStyle.ACADEMIC,
enable_deep_thinking=False,
)
events = []
async for event in generator:
events.append(event)
assert len(events) == 1
assert "event: tool_calls" in events[0]
assert "Making tool call" in events[0]
assert "tool_calls" in events[0]
@pytest.mark.asyncio
@patch("src.server.app.graph")
async def test_astream_workflow_generator_ai_message_with_tool_call_chunks(
self, mock_graph
):
# Mock AI message with only tool call chunks
mock_ai_message = AIMessageChunk(content="Streaming tool call")
mock_ai_message.id = "msg_101"
mock_ai_message.response_metadata = {}
mock_ai_message.tool_calls = []
mock_ai_message.tool_call_chunks = [{"name": "search", "index": 0}]
async def mock_astream(*args, **kwargs):
yield ("agent1:subagent", "step1", (mock_ai_message, {}))
mock_graph.astream = mock_astream
generator = _astream_workflow_generator(
messages=[],
thread_id="test_thread",
resources=[],
max_plan_iterations=3,
max_step_num=10,
max_search_results=5,
auto_accepted_plan=True,
interrupt_feedback="",
mcp_settings={},
enable_background_investigation=False,
report_style=ReportStyle.ACADEMIC,
enable_deep_thinking=False,
)
events = []
async for event in generator:
events.append(event)
assert len(events) == 1
assert "event: tool_call_chunks" in events[0]
assert "Streaming tool call" in events[0]
@pytest.mark.asyncio
@patch("src.server.app.graph")
async def test_astream_workflow_generator_with_finish_reason(self, mock_graph):
# Mock AI message with finish reason
mock_ai_message = AIMessageChunk(content="Complete response")
mock_ai_message.id = "msg_finish"
mock_ai_message.response_metadata = {"finish_reason": "stop"}
mock_ai_message.tool_calls = []
mock_ai_message.tool_call_chunks = []
async def mock_astream(*args, **kwargs):
yield ("agent1:subagent", "step1", (mock_ai_message, {}))
mock_graph.astream = mock_astream
generator = _astream_workflow_generator(
messages=[],
thread_id="test_thread",
resources=[],
max_plan_iterations=3,
max_step_num=10,
max_search_results=5,
auto_accepted_plan=True,
interrupt_feedback="",
mcp_settings={},
enable_background_investigation=False,
report_style=ReportStyle.ACADEMIC,
enable_deep_thinking=False,
)
events = []
async for event in generator:
events.append(event)
assert len(events) == 1
assert "event: message_chunk" in events[0]
assert "finish_reason" in events[0]
assert "stop" in events[0]
@pytest.mark.asyncio
@patch("src.server.app.graph")
async def test_astream_workflow_generator_config_passed_correctly(self, mock_graph):
mock_ai_message = AIMessageChunk(content="Test")
mock_ai_message.id = "test_id"
mock_ai_message.response_metadata = {}
mock_ai_message.tool_calls = []
mock_ai_message.tool_call_chunks = []
async def verify_config(*args, **kwargs):
config = kwargs.get("config", {})
assert config["thread_id"] == "test_thread"
assert config["max_plan_iterations"] == 5
assert config["max_step_num"] == 20
assert config["max_search_results"] == 10
assert config["report_style"] == ReportStyle.NEWS.value
yield ("agent1", "messages", [mock_ai_message])
class TestGenerateProseEndpoint:
@patch("src.server.app.build_prose_graph")
def test_generate_prose_success(self, mock_build_graph, client):
# Mock the workflow and its astream method
mock_workflow = MagicMock()
mock_build_graph.return_value = mock_workflow
class MockEvent:
def __init__(self, content):
self.content = content
async def mock_astream(*args, **kwargs):
yield (None, [MockEvent("Generated prose 1")])
yield (None, [MockEvent("Generated prose 2")])
mock_workflow.astream.return_value = mock_astream()
request_data = {
"prompt": "Write a story.",
"option": "default",
"command": "generate",
}
response = client.post("/api/prose/generate", json=request_data)
assert response.status_code == 200
assert response.headers["content-type"].startswith("text/event-stream")
# Read the streaming response content
content = b"".join(response.iter_bytes())
assert b"Generated prose 1" in content or b"Generated prose 2" in content
@patch("src.server.app.build_prose_graph")
def test_generate_prose_error(self, mock_build_graph, client):
mock_build_graph.side_effect = Exception("Prose generation failed")
request_data = {
"prompt": "Write a story.",
"option": "default",
"command": "generate",
}
response = client.post("/api/prose/generate", json=request_data)
assert response.status_code == 500
assert response.json()["detail"] == "Internal Server Error"
+167
View File
@@ -0,0 +1,167 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from pydantic import ValidationError
from src.config.report_style import ReportStyle
from src.rag.retriever import Resource
from unittest.mock import AsyncMock, patch, MagicMock
from fastapi import HTTPException
from src.server.chat_request import (
ContentItem,
ChatMessage,
ChatRequest,
TTSRequest,
GeneratePodcastRequest,
GeneratePPTRequest,
GenerateProseRequest,
EnhancePromptRequest,
)
import src.server.mcp_utils as mcp_utils # Assuming mcp_utils is the module to test
def test_content_item_text_and_image():
item_text = ContentItem(type="text", text="hello")
assert item_text.type == "text"
assert item_text.text == "hello"
assert item_text.image_url is None
item_image = ContentItem(type="image", image_url="http://img.com/1.png")
assert item_image.type == "image"
assert item_image.text is None
assert item_image.image_url == "http://img.com/1.png"
def test_chat_message_with_string_content():
msg = ChatMessage(role="user", content="Hello!")
assert msg.role == "user"
assert msg.content == "Hello!"
def test_chat_message_with_content_items():
items = [ContentItem(type="text", text="hi")]
msg = ChatMessage(role="assistant", content=items)
assert msg.role == "assistant"
assert isinstance(msg.content, list)
assert msg.content[0].type == "text"
def test_chat_request_defaults():
req = ChatRequest()
assert req.messages == []
assert req.resources == []
assert req.debug is False
assert req.thread_id == "__default__"
assert req.max_plan_iterations == 1
assert req.max_step_num == 3
assert req.max_search_results == 3
assert req.auto_accepted_plan is False
assert req.interrupt_feedback is None
assert req.mcp_settings is None
assert req.enable_background_investigation is True
assert req.report_style == ReportStyle.ACADEMIC
def test_chat_request_with_values():
resource = Resource(
name="test", type="doc", uri="some-uri-value", title="some-title-value"
)
msg = ChatMessage(role="user", content="hi")
req = ChatRequest(
messages=[msg],
resources=[resource],
debug=True,
thread_id="tid",
max_plan_iterations=2,
max_step_num=5,
max_search_results=10,
auto_accepted_plan=True,
interrupt_feedback="stop",
mcp_settings={"foo": "bar"},
enable_background_investigation=False,
report_style="academic",
)
assert req.messages[0].role == "user"
assert req.debug is True
assert req.thread_id == "tid"
assert req.max_plan_iterations == 2
assert req.max_step_num == 5
assert req.max_search_results == 10
assert req.auto_accepted_plan is True
assert req.interrupt_feedback == "stop"
assert req.mcp_settings == {"foo": "bar"}
assert req.enable_background_investigation is False
assert req.report_style == ReportStyle.ACADEMIC
def test_tts_request_defaults():
req = TTSRequest(text="hello")
assert req.text == "hello"
assert req.voice_type == "BV700_V2_streaming"
assert req.encoding == "mp3"
assert req.speed_ratio == 1.0
assert req.volume_ratio == 1.0
assert req.pitch_ratio == 1.0
assert req.text_type == "plain"
assert req.with_frontend == 1
assert req.frontend_type == "unitTson"
def test_generate_podcast_request():
req = GeneratePodcastRequest(content="Podcast content")
assert req.content == "Podcast content"
def test_generate_ppt_request():
req = GeneratePPTRequest(content="PPT content")
assert req.content == "PPT content"
def test_generate_prose_request():
req = GenerateProseRequest(prompt="Write a poem", option="poet", command="rhyme")
assert req.prompt == "Write a poem"
assert req.option == "poet"
assert req.command == "rhyme"
req2 = GenerateProseRequest(prompt="Write", option="short")
assert req2.command == ""
def test_enhance_prompt_request_defaults():
req = EnhancePromptRequest(prompt="Improve this")
assert req.prompt == "Improve this"
assert req.context == ""
assert req.report_style == "academic"
def test_content_item_validation_error():
with pytest.raises(ValidationError):
ContentItem() # missing required 'type'
def test_chat_message_validation_error():
with pytest.raises(ValidationError):
ChatMessage(role="user") # missing content
def test_tts_request_validation_error():
with pytest.raises(ValidationError):
TTSRequest() # missing required 'text'
@pytest.mark.asyncio
@patch("src.server.mcp_utils._get_tools_from_client_session", new_callable=AsyncMock)
@patch("src.server.mcp_utils.StdioServerParameters")
@patch("src.server.mcp_utils.stdio_client")
async def test_load_mcp_tools_exception_handling(
mock_stdio_client, mock_StdioServerParameters, mock_get_tools
): # Changed to async def
mock_get_tools.side_effect = Exception("unexpected error")
mock_StdioServerParameters.return_value = MagicMock()
mock_stdio_client.return_value = MagicMock()
with pytest.raises(HTTPException) as exc:
await mcp_utils.load_mcp_tools(server_type="stdio", command="foo") # Use await
assert exc.value.status_code == 500
assert "unexpected error" in exc.value.detail
+73
View File
@@ -0,0 +1,73 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from pydantic import ValidationError
from src.server.mcp_request import MCPServerMetadataRequest, MCPServerMetadataResponse
def test_mcp_server_metadata_request_required_fields():
# 'transport' is required
req = MCPServerMetadataRequest(transport="stdio")
assert req.transport == "stdio"
assert req.command is None
assert req.args is None
assert req.url is None
assert req.env is None
assert req.timeout_seconds is None
def test_mcp_server_metadata_request_optional_fields():
req = MCPServerMetadataRequest(
transport="sse",
command="run",
args=["--foo", "bar"],
url="http://localhost:8080",
env={"FOO": "BAR"},
timeout_seconds=30,
)
assert req.transport == "sse"
assert req.command == "run"
assert req.args == ["--foo", "bar"]
assert req.url == "http://localhost:8080"
assert req.env == {"FOO": "BAR"}
assert req.timeout_seconds == 30
def test_mcp_server_metadata_request_missing_transport():
with pytest.raises(ValidationError):
MCPServerMetadataRequest()
def test_mcp_server_metadata_response_required_fields():
resp = MCPServerMetadataResponse(transport="stdio")
assert resp.transport == "stdio"
assert resp.command is None
assert resp.args is None
assert resp.url is None
assert resp.env is None
assert resp.tools == []
def test_mcp_server_metadata_response_optional_fields():
resp = MCPServerMetadataResponse(
transport="sse",
command="run",
args=["--foo", "bar"],
url="http://localhost:8080",
env={"FOO": "BAR"},
tools=["tool1", "tool2"],
)
assert resp.transport == "sse"
assert resp.command == "run"
assert resp.args == ["--foo", "bar"]
assert resp.url == "http://localhost:8080"
assert resp.env == {"FOO": "BAR"}
assert resp.tools == ["tool1", "tool2"]
def test_mcp_server_metadata_response_tools_default_factory():
resp1 = MCPServerMetadataResponse(transport="stdio")
resp2 = MCPServerMetadataResponse(transport="stdio")
resp1.tools.append("toolA")
assert resp2.tools == [] # Should not share list between instances
+121
View File
@@ -0,0 +1,121 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from unittest.mock import AsyncMock, patch, MagicMock
from fastapi import HTTPException
import src.server.mcp_utils as mcp_utils
@pytest.mark.asyncio
@patch("src.server.mcp_utils.ClientSession")
async def test__get_tools_from_client_session_success(mock_ClientSession):
mock_read = AsyncMock()
mock_write = AsyncMock()
mock_context_manager = AsyncMock()
mock_context_manager.__aenter__.return_value = (mock_read, mock_write)
mock_context_manager.__aexit__.return_value = None
mock_session = AsyncMock()
mock_session.__aenter__.return_value = mock_session
mock_session.__aexit__.return_value = None
mock_session.initialize = AsyncMock()
mock_tools_obj = MagicMock()
mock_tools_obj.tools = ["tool1", "tool2"]
mock_session.list_tools = AsyncMock(return_value=mock_tools_obj)
mock_ClientSession.return_value = mock_session
result = await mcp_utils._get_tools_from_client_session(
mock_context_manager, timeout_seconds=5
)
assert result == ["tool1", "tool2"]
mock_session.initialize.assert_awaited_once()
mock_session.list_tools.assert_awaited_once()
@pytest.mark.asyncio
@patch("src.server.mcp_utils._get_tools_from_client_session", new_callable=AsyncMock)
@patch("src.server.mcp_utils.StdioServerParameters")
@patch("src.server.mcp_utils.stdio_client")
async def test_load_mcp_tools_stdio_success(
mock_stdio_client, mock_StdioServerParameters, mock_get_tools
):
mock_get_tools.return_value = ["toolA"]
params = MagicMock()
mock_StdioServerParameters.return_value = params
mock_client = MagicMock()
mock_stdio_client.return_value = mock_client
result = await mcp_utils.load_mcp_tools(
server_type="stdio",
command="echo",
args=["foo"],
env={"FOO": "BAR"},
timeout_seconds=3,
)
assert result == ["toolA"]
mock_StdioServerParameters.assert_called_once_with(
command="echo", args=["foo"], env={"FOO": "BAR"}
)
mock_stdio_client.assert_called_once_with(params)
mock_get_tools.assert_awaited_once_with(mock_client, 3)
@pytest.mark.asyncio
async def test_load_mcp_tools_stdio_missing_command():
with pytest.raises(HTTPException) as exc:
await mcp_utils.load_mcp_tools(server_type="stdio")
assert exc.value.status_code == 400
assert "Command is required" in exc.value.detail
@pytest.mark.asyncio
@patch("src.server.mcp_utils._get_tools_from_client_session", new_callable=AsyncMock)
@patch("src.server.mcp_utils.sse_client")
async def test_load_mcp_tools_sse_success(mock_sse_client, mock_get_tools):
mock_get_tools.return_value = ["toolB"]
mock_client = MagicMock()
mock_sse_client.return_value = mock_client
result = await mcp_utils.load_mcp_tools(
server_type="sse",
url="http://localhost:1234",
timeout_seconds=7,
)
assert result == ["toolB"]
mock_sse_client.assert_called_once_with(url="http://localhost:1234")
mock_get_tools.assert_awaited_once_with(mock_client, 7)
@pytest.mark.asyncio
async def test_load_mcp_tools_sse_missing_url():
with pytest.raises(HTTPException) as exc:
await mcp_utils.load_mcp_tools(server_type="sse")
assert exc.value.status_code == 400
assert "URL is required" in exc.value.detail
@pytest.mark.asyncio
async def test_load_mcp_tools_unsupported_type():
with pytest.raises(HTTPException) as exc:
await mcp_utils.load_mcp_tools(server_type="unknown")
assert exc.value.status_code == 400
assert "Unsupported server type" in exc.value.detail
@pytest.mark.asyncio
@patch("src.server.mcp_utils._get_tools_from_client_session", new_callable=AsyncMock)
@patch("src.server.mcp_utils.StdioServerParameters")
@patch("src.server.mcp_utils.stdio_client")
async def test_load_mcp_tools_exception_handling(
mock_stdio_client, mock_StdioServerParameters, mock_get_tools
):
mock_get_tools.side_effect = Exception("unexpected error")
mock_StdioServerParameters.return_value = MagicMock()
mock_stdio_client.return_value = MagicMock()
with pytest.raises(HTTPException) as exc:
await mcp_utils.load_mcp_tools(server_type="stdio", command="foo")
assert exc.value.status_code == 500
assert "unexpected error" in exc.value.detail
+109
View File
@@ -0,0 +1,109 @@
from unittest.mock import Mock, patch
from src.tools.crawl import crawl_tool
class TestCrawlTool:
@patch("src.tools.crawl.Crawler")
def test_crawl_tool_success(self, mock_crawler_class):
# Arrange
mock_crawler = Mock()
mock_article = Mock()
mock_article.to_markdown.return_value = (
"# Test Article\nThis is test content." * 100
)
mock_crawler.crawl.return_value = mock_article
mock_crawler_class.return_value = mock_crawler
url = "https://example.com"
# Act
result = crawl_tool(url)
# Assert
assert isinstance(result, dict)
assert result["url"] == url
assert "crawled_content" in result
assert len(result["crawled_content"]) <= 1000
mock_crawler_class.assert_called_once()
mock_crawler.crawl.assert_called_once_with(url)
mock_article.to_markdown.assert_called_once()
@patch("src.tools.crawl.Crawler")
def test_crawl_tool_short_content(self, mock_crawler_class):
# Arrange
mock_crawler = Mock()
mock_article = Mock()
short_content = "Short content"
mock_article.to_markdown.return_value = short_content
mock_crawler.crawl.return_value = mock_article
mock_crawler_class.return_value = mock_crawler
url = "https://example.com"
# Act
result = crawl_tool(url)
# Assert
assert result["crawled_content"] == short_content
@patch("src.tools.crawl.Crawler")
@patch("src.tools.crawl.logger")
def test_crawl_tool_crawler_exception(self, mock_logger, mock_crawler_class):
# Arrange
mock_crawler = Mock()
mock_crawler.crawl.side_effect = Exception("Network error")
mock_crawler_class.return_value = mock_crawler
url = "https://example.com"
# Act
result = crawl_tool(url)
# Assert
assert isinstance(result, str)
assert "Failed to crawl" in result
assert "Network error" in result
mock_logger.error.assert_called_once()
@patch("src.tools.crawl.Crawler")
@patch("src.tools.crawl.logger")
def test_crawl_tool_crawler_instantiation_exception(
self, mock_logger, mock_crawler_class
):
# Arrange
mock_crawler_class.side_effect = Exception("Crawler init error")
url = "https://example.com"
# Act
result = crawl_tool(url)
# Assert
assert isinstance(result, str)
assert "Failed to crawl" in result
assert "Crawler init error" in result
mock_logger.error.assert_called_once()
@patch("src.tools.crawl.Crawler")
@patch("src.tools.crawl.logger")
def test_crawl_tool_markdown_conversion_exception(
self, mock_logger, mock_crawler_class
):
# Arrange
mock_crawler = Mock()
mock_article = Mock()
mock_article.to_markdown.side_effect = Exception("Markdown conversion error")
mock_crawler.crawl.return_value = mock_article
mock_crawler_class.return_value = mock_crawler
url = "https://example.com"
# Act
result = crawl_tool(url)
# Assert
assert isinstance(result, str)
assert "Failed to crawl" in result
assert "Markdown conversion error" in result
mock_logger.error.assert_called_once()
+119
View File
@@ -0,0 +1,119 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
from unittest.mock import Mock, call, patch
from src.tools.decorators import create_logged_tool
class MockBaseTool:
"""Mock base tool class for testing."""
def _run(self, *args, **kwargs):
return "base_result"
class TestLoggedToolMixin:
def test_run_calls_log_operation(self):
"""Test that _run calls _log_operation with correct parameters."""
# Create a logged tool instance
LoggedTool = create_logged_tool(MockBaseTool)
tool = LoggedTool()
# Mock the _log_operation method
tool._log_operation = Mock()
# Call _run with test parameters
args = ("arg1", "arg2")
kwargs = {"key1": "value1", "key2": "value2"}
tool._run(*args, **kwargs)
# Verify _log_operation was called with correct parameters
tool._log_operation.assert_called_once_with("_run", *args, **kwargs)
def test_run_calls_super_run(self):
"""Test that _run calls the parent class _run method."""
# Create a logged tool instance
LoggedTool = create_logged_tool(MockBaseTool)
tool = LoggedTool()
# Mock the parent _run method
with patch.object(
MockBaseTool, "_run", return_value="mocked_result"
) as mock_super_run:
args = ("arg1", "arg2")
kwargs = {"key1": "value1"}
result = tool._run(*args, **kwargs)
# Verify super()._run was called with correct parameters
mock_super_run.assert_called_once_with(*args, **kwargs)
# Verify the result is returned
assert result == "mocked_result"
def test_run_logs_result(self):
"""Test that _run logs the result with debug level."""
LoggedTool = create_logged_tool(MockBaseTool)
tool = LoggedTool()
with patch("src.tools.decorators.logger.debug") as mock_debug:
tool._run("test_arg")
# Verify debug log was called with correct message
mock_debug.assert_has_calls(
[
call("Tool MockBaseTool._run called with parameters: test_arg"),
call("Tool MockBaseTool returned: base_result"),
]
)
def test_run_returns_super_result(self):
"""Test that _run returns the result from parent class."""
LoggedTool = create_logged_tool(MockBaseTool)
tool = LoggedTool()
result = tool._run()
assert result == "base_result"
def test_run_with_no_args(self):
"""Test _run method with no arguments."""
LoggedTool = create_logged_tool(MockBaseTool)
tool = LoggedTool()
with patch("src.tools.decorators.logger.debug") as mock_debug:
tool._log_operation = Mock()
result = tool._run()
# Verify _log_operation called with no args
tool._log_operation.assert_called_once_with("_run")
# Verify result logging
mock_debug.assert_called_once()
assert result == "base_result"
def test_run_with_mixed_args_kwargs(self):
"""Test _run method with both positional and keyword arguments."""
LoggedTool = create_logged_tool(MockBaseTool)
tool = LoggedTool()
tool._log_operation = Mock()
args = ("pos1", "pos2")
kwargs = {"kw1": "val1", "kw2": "val2"}
result = tool._run(*args, **kwargs)
# Verify all arguments passed correctly
tool._log_operation.assert_called_once_with("_run", *args, **kwargs)
assert result == "base_result"
def test_run_class_name_replacement(self):
"""Test that class name 'Logged' prefix is correctly removed in logging."""
LoggedTool = create_logged_tool(MockBaseTool)
tool = LoggedTool()
with patch("src.tools.decorators.logger.debug") as mock_debug:
tool._run()
# Verify the logged class name has 'Logged' prefix removed
call_args = mock_debug.call_args[0][0]
assert "Tool MockBaseTool returned:" in call_args
assert "LoggedMockBaseTool" not in call_args
+147
View File
@@ -0,0 +1,147 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import pytest
from unittest.mock import patch
from src.tools.python_repl import python_repl_tool
class TestPythonReplTool:
@patch("src.tools.python_repl.repl")
@patch("src.tools.python_repl.logger")
def test_successful_code_execution(self, mock_logger, mock_repl):
# Arrange
code = "print('Hello, World!')"
expected_output = "Hello, World!\n"
mock_repl.run.return_value = expected_output
# Act
result = python_repl_tool(code)
# Assert
mock_repl.run.assert_called_once_with(code)
mock_logger.info.assert_called_with("Code execution successful")
assert "Successfully executed:" in result
assert code in result
assert expected_output in result
@patch("src.tools.python_repl.repl")
@patch("src.tools.python_repl.logger")
def test_invalid_input_type(self, mock_logger, mock_repl):
# Arrange
invalid_code = 123
# Act & Assert - expect ValidationError from LangChain
with pytest.raises(Exception) as exc_info:
python_repl_tool(invalid_code)
# Verify that it's a validation error
assert "ValidationError" in str(
type(exc_info.value)
) or "validation error" in str(exc_info.value)
# The REPL should not be called since validation fails first
mock_repl.run.assert_not_called()
@patch("src.tools.python_repl.repl")
@patch("src.tools.python_repl.logger")
def test_code_execution_with_error_in_result(self, mock_logger, mock_repl):
# Arrange
code = "invalid_function()"
error_result = "NameError: name 'invalid_function' is not defined"
mock_repl.run.return_value = error_result
# Act
result = python_repl_tool(code)
# Assert
mock_repl.run.assert_called_once_with(code)
mock_logger.error.assert_called_with(error_result)
assert "Error executing code:" in result
assert code in result
assert error_result in result
@patch("src.tools.python_repl.repl")
@patch("src.tools.python_repl.logger")
def test_code_execution_with_exception_in_result(self, mock_logger, mock_repl):
# Arrange
code = "1/0"
exception_result = "ZeroDivisionError: division by zero"
mock_repl.run.return_value = exception_result
# Act
result = python_repl_tool(code)
# Assert
mock_repl.run.assert_called_once_with(code)
mock_logger.error.assert_called_with(exception_result)
assert "Error executing code:" in result
assert code in result
assert exception_result in result
@patch("src.tools.python_repl.repl")
@patch("src.tools.python_repl.logger")
def test_code_execution_raises_exception(self, mock_logger, mock_repl):
# Arrange
code = "print('test')"
exception = RuntimeError("REPL failed")
mock_repl.run.side_effect = exception
# Act
result = python_repl_tool(code)
# Assert
mock_repl.run.assert_called_once_with(code)
mock_logger.error.assert_called_with(repr(exception))
assert "Error executing code:" in result
assert code in result
assert repr(exception) in result
@patch("src.tools.python_repl.repl")
@patch("src.tools.python_repl.logger")
def test_successful_execution_with_calculation(self, mock_logger, mock_repl):
# Arrange
code = "result = 2 + 3\nprint(result)"
expected_output = "5\n"
mock_repl.run.return_value = expected_output
# Act
result = python_repl_tool(code)
# Assert
mock_repl.run.assert_called_once_with(code)
mock_logger.info.assert_any_call("Executing Python code")
mock_logger.info.assert_any_call("Code execution successful")
assert "Successfully executed:" in result
assert code in result
assert expected_output in result
@patch("src.tools.python_repl.repl")
@patch("src.tools.python_repl.logger")
def test_empty_string_code(self, mock_logger, mock_repl):
# Arrange
code = ""
mock_repl.run.return_value = ""
# Act
result = python_repl_tool(code)
# Assert
mock_repl.run.assert_called_once_with(code)
mock_logger.info.assert_called_with("Code execution successful")
assert "Successfully executed:" in result
@patch("src.tools.python_repl.repl")
@patch("src.tools.python_repl.logger")
def test_logging_calls(self, mock_logger, mock_repl):
# Arrange
code = "x = 1"
mock_repl.run.return_value = ""
# Act
python_repl_tool(code)
# Assert
mock_logger.info.assert_any_call("Executing Python code")
mock_logger.info.assert_any_call("Code execution successful")
+54
View File
@@ -0,0 +1,54 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import os
import pytest
from unittest.mock import patch
from src.tools.search import get_web_search_tool
from src.config import SearchEngine
class TestGetWebSearchTool:
@patch("src.tools.search.SELECTED_SEARCH_ENGINE", SearchEngine.TAVILY.value)
def test_get_web_search_tool_tavily(self):
tool = get_web_search_tool(max_search_results=5)
assert tool.name == "web_search"
assert tool.max_results == 5
assert tool.include_raw_content is True
assert tool.include_images is True
assert tool.include_image_descriptions is True
@patch("src.tools.search.SELECTED_SEARCH_ENGINE", SearchEngine.DUCKDUCKGO.value)
def test_get_web_search_tool_duckduckgo(self):
tool = get_web_search_tool(max_search_results=3)
assert tool.name == "web_search"
assert tool.max_results == 3
@patch("src.tools.search.SELECTED_SEARCH_ENGINE", SearchEngine.BRAVE_SEARCH.value)
@patch.dict(os.environ, {"BRAVE_SEARCH_API_KEY": "test_api_key"})
def test_get_web_search_tool_brave(self):
tool = get_web_search_tool(max_search_results=4)
assert tool.name == "web_search"
assert tool.search_wrapper.api_key == "test_api_key"
@patch("src.tools.search.SELECTED_SEARCH_ENGINE", SearchEngine.ARXIV.value)
def test_get_web_search_tool_arxiv(self):
tool = get_web_search_tool(max_search_results=2)
assert tool.name == "web_search"
assert tool.api_wrapper.top_k_results == 2
assert tool.api_wrapper.load_max_docs == 2
assert tool.api_wrapper.load_all_available_meta is True
@patch("src.tools.search.SELECTED_SEARCH_ENGINE", "unsupported_engine")
def test_get_web_search_tool_unsupported_engine(self):
with pytest.raises(
ValueError, match="Unsupported search engine: unsupported_engine"
):
get_web_search_tool(max_search_results=1)
@patch("src.tools.search.SELECTED_SEARCH_ENGINE", SearchEngine.BRAVE_SEARCH.value)
@patch.dict(os.environ, {}, clear=True)
def test_get_web_search_tool_brave_no_api_key(self):
tool = get_web_search_tool(max_search_results=1)
assert tool.search_wrapper.api_key == ""
@@ -0,0 +1,206 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import json
import pytest
from unittest.mock import Mock, patch, AsyncMock, MagicMock
import requests
from src.tools.tavily_search.tavily_search_api_wrapper import (
EnhancedTavilySearchAPIWrapper,
)
class TestEnhancedTavilySearchAPIWrapper:
@pytest.fixture
def wrapper(self):
with patch(
"src.tools.tavily_search.tavily_search_api_wrapper.OriginalTavilySearchAPIWrapper"
):
wrapper = EnhancedTavilySearchAPIWrapper(tavily_api_key="dummy-key")
# The parent class is mocked, so initialization won't fail
return wrapper
@pytest.fixture
def mock_response_data(self):
return {
"results": [
{
"title": "Test Title",
"url": "https://example.com",
"content": "Test content",
"score": 0.9,
"raw_content": "Raw test content",
}
],
"images": [
{
"url": "https://example.com/image.jpg",
"description": "Test image description",
}
],
}
@patch("src.tools.tavily_search.tavily_search_api_wrapper.requests.post")
def test_raw_results_success(self, mock_post, wrapper, mock_response_data):
mock_response = Mock()
mock_response.json.return_value = mock_response_data
mock_response.raise_for_status.return_value = None
mock_post.return_value = mock_response
result = wrapper.raw_results("test query", max_results=10)
assert result == mock_response_data
mock_post.assert_called_once()
call_args = mock_post.call_args
assert "json" in call_args.kwargs
assert call_args.kwargs["json"]["query"] == "test query"
assert call_args.kwargs["json"]["max_results"] == 10
@patch("src.tools.tavily_search.tavily_search_api_wrapper.requests.post")
def test_raw_results_with_all_parameters(
self, mock_post, wrapper, mock_response_data
):
mock_response = Mock()
mock_response.json.return_value = mock_response_data
mock_response.raise_for_status.return_value = None
mock_post.return_value = mock_response
result = wrapper.raw_results(
"test query",
max_results=3,
search_depth="basic",
include_domains=["example.com"],
exclude_domains=["spam.com"],
include_answer=True,
include_raw_content=True,
include_images=True,
include_image_descriptions=True,
)
assert result == mock_response_data
call_args = mock_post.call_args
params = call_args.kwargs["json"]
assert params["include_domains"] == ["example.com"]
assert params["exclude_domains"] == ["spam.com"]
assert params["include_answer"] is True
assert params["include_raw_content"] is True
@patch("src.tools.tavily_search.tavily_search_api_wrapper.requests.post")
def test_raw_results_http_error(self, mock_post, wrapper):
mock_response = Mock()
mock_response.raise_for_status.side_effect = requests.HTTPError("API Error")
mock_post.return_value = mock_response
with pytest.raises(requests.HTTPError):
wrapper.raw_results("test query")
@pytest.mark.asyncio
async def test_raw_results_async_success(self, wrapper, mock_response_data):
# Create a mock that acts as both the response and its context manager
mock_response_cm = AsyncMock()
mock_response_cm.__aenter__ = AsyncMock(return_value=mock_response_cm)
mock_response_cm.__aexit__ = AsyncMock(return_value=None)
mock_response_cm.status = 200
mock_response_cm.text = AsyncMock(return_value=json.dumps(mock_response_data))
# Create mock session that returns the context manager
mock_session = AsyncMock()
mock_session.post = MagicMock(
return_value=mock_response_cm
) # Use MagicMock, not AsyncMock
# Create mock session class
mock_session_cm = AsyncMock()
mock_session_cm.__aenter__ = AsyncMock(return_value=mock_session)
mock_session_cm.__aexit__ = AsyncMock(return_value=None)
with patch(
"src.tools.tavily_search.tavily_search_api_wrapper.aiohttp.ClientSession",
return_value=mock_session_cm,
):
result = await wrapper.raw_results_async("test query")
assert result == mock_response_data
@pytest.mark.asyncio
async def test_raw_results_async_error(self, wrapper):
# Create a mock that acts as both the response and its context manager
mock_response_cm = AsyncMock()
mock_response_cm.__aenter__ = AsyncMock(return_value=mock_response_cm)
mock_response_cm.__aexit__ = AsyncMock(return_value=None)
mock_response_cm.status = 400
mock_response_cm.reason = "Bad Request"
# Create mock session that returns the context manager
mock_session = AsyncMock()
mock_session.post = MagicMock(
return_value=mock_response_cm
) # Use MagicMock, not AsyncMock
# Create mock session class
mock_session_cm = AsyncMock()
mock_session_cm.__aenter__ = AsyncMock(return_value=mock_session)
mock_session_cm.__aexit__ = AsyncMock(return_value=None)
with patch(
"src.tools.tavily_search.tavily_search_api_wrapper.aiohttp.ClientSession",
return_value=mock_session_cm,
):
with pytest.raises(Exception, match="Error 400: Bad Request"):
await wrapper.raw_results_async("test query")
def test_clean_results_with_images(self, wrapper, mock_response_data):
result = wrapper.clean_results_with_images(mock_response_data)
assert len(result) == 2
# Test page result
page_result = result[0]
assert page_result["type"] == "page"
assert page_result["title"] == "Test Title"
assert page_result["url"] == "https://example.com"
assert page_result["content"] == "Test content"
assert page_result["score"] == 0.9
assert page_result["raw_content"] == "Raw test content"
# Test image result
image_result = result[1]
assert image_result["type"] == "image"
assert image_result["image_url"] == "https://example.com/image.jpg"
assert image_result["image_description"] == "Test image description"
def test_clean_results_without_raw_content(self, wrapper):
data = {
"results": [
{
"title": "Test Title",
"url": "https://example.com",
"content": "Test content",
"score": 0.9,
}
],
"images": [],
}
result = wrapper.clean_results_with_images(data)
assert len(result) == 1
assert "raw_content" not in result[0]
def test_clean_results_empty_images(self, wrapper):
data = {
"results": [
{
"title": "Test Title",
"url": "https://example.com",
"content": "Test content",
"score": 0.9,
}
],
"images": [],
}
result = wrapper.clean_results_with_images(data)
assert len(result) == 1
assert result[0]["type"] == "page"
@@ -0,0 +1,264 @@
# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates
# SPDX-License-Identifier: MIT
import json
import pytest
from unittest.mock import Mock, patch, AsyncMock
from src.tools.tavily_search.tavily_search_results_with_images import (
TavilySearchResultsWithImages,
)
from src.tools.tavily_search.tavily_search_api_wrapper import (
EnhancedTavilySearchAPIWrapper,
)
class TestTavilySearchResultsWithImages:
@pytest.fixture
def mock_api_wrapper(self):
"""Create a mock API wrapper."""
wrapper = Mock(spec=EnhancedTavilySearchAPIWrapper)
return wrapper
@pytest.fixture
def search_tool(self, mock_api_wrapper):
"""Create a TavilySearchResultsWithImages instance with mocked dependencies."""
tool = TavilySearchResultsWithImages(
max_results=5,
include_answer=True,
include_raw_content=True,
include_images=True,
include_image_descriptions=True,
)
tool.api_wrapper = mock_api_wrapper
return tool
@pytest.fixture
def sample_raw_results(self):
"""Sample raw results from Tavily API."""
return {
"query": "test query",
"answer": "Test answer",
"images": ["https://example.com/image1.jpg"],
"results": [
{
"title": "Test Title",
"url": "https://example.com",
"content": "Test content",
"score": 0.95,
"raw_content": "Raw test content",
}
],
"response_time": 1.5,
}
@pytest.fixture
def sample_cleaned_results(self):
"""Sample cleaned results."""
return [
{
"title": "Test Title",
"url": "https://example.com",
"content": "Test content",
}
]
def test_init_default_values(self):
"""Test initialization with default values."""
tool = TavilySearchResultsWithImages()
assert tool.include_image_descriptions is False
assert isinstance(tool.api_wrapper, EnhancedTavilySearchAPIWrapper)
def test_init_custom_values(self):
"""Test initialization with custom values."""
tool = TavilySearchResultsWithImages(
max_results=10, include_image_descriptions=True
)
assert tool.max_results == 10
assert tool.include_image_descriptions is True
@patch("builtins.print")
def test_run_success(
self,
mock_print,
search_tool,
mock_api_wrapper,
sample_raw_results,
sample_cleaned_results,
):
"""Test successful synchronous run."""
mock_api_wrapper.raw_results.return_value = sample_raw_results
mock_api_wrapper.clean_results_with_images.return_value = sample_cleaned_results
result, raw = search_tool._run("test query")
assert result == sample_cleaned_results
assert raw == sample_raw_results
mock_api_wrapper.raw_results.assert_called_once_with(
"test query",
search_tool.max_results,
search_tool.search_depth,
search_tool.include_domains,
search_tool.exclude_domains,
search_tool.include_answer,
search_tool.include_raw_content,
search_tool.include_images,
search_tool.include_image_descriptions,
)
mock_api_wrapper.clean_results_with_images.assert_called_once_with(
sample_raw_results
)
mock_print.assert_called_once()
@patch("builtins.print")
def test_run_exception(self, mock_print, search_tool, mock_api_wrapper):
"""Test synchronous run with exception."""
mock_api_wrapper.raw_results.side_effect = Exception("API Error")
result, raw = search_tool._run("test query")
assert "API Error" in result
assert raw == {}
mock_api_wrapper.clean_results_with_images.assert_not_called()
@pytest.mark.asyncio
@patch("builtins.print")
async def test_arun_success(
self,
mock_print,
search_tool,
mock_api_wrapper,
sample_raw_results,
sample_cleaned_results,
):
"""Test successful asynchronous run."""
mock_api_wrapper.raw_results_async = AsyncMock(return_value=sample_raw_results)
mock_api_wrapper.clean_results_with_images.return_value = sample_cleaned_results
result, raw = await search_tool._arun("test query")
assert result == sample_cleaned_results
assert raw == sample_raw_results
mock_api_wrapper.raw_results_async.assert_called_once_with(
"test query",
search_tool.max_results,
search_tool.search_depth,
search_tool.include_domains,
search_tool.exclude_domains,
search_tool.include_answer,
search_tool.include_raw_content,
search_tool.include_images,
search_tool.include_image_descriptions,
)
mock_api_wrapper.clean_results_with_images.assert_called_once_with(
sample_raw_results
)
mock_print.assert_called_once()
@pytest.mark.asyncio
@patch("builtins.print")
async def test_arun_exception(self, mock_print, search_tool, mock_api_wrapper):
"""Test asynchronous run with exception."""
mock_api_wrapper.raw_results_async = AsyncMock(
side_effect=Exception("Async API Error")
)
result, raw = await search_tool._arun("test query")
assert "Async API Error" in result
assert raw == {}
mock_api_wrapper.clean_results_with_images.assert_not_called()
@patch("builtins.print")
def test_run_with_run_manager(
self,
mock_print,
search_tool,
mock_api_wrapper,
sample_raw_results,
sample_cleaned_results,
):
"""Test run with callback manager."""
mock_run_manager = Mock()
mock_api_wrapper.raw_results.return_value = sample_raw_results
mock_api_wrapper.clean_results_with_images.return_value = sample_cleaned_results
result, raw = search_tool._run("test query", run_manager=mock_run_manager)
assert result == sample_cleaned_results
assert raw == sample_raw_results
@pytest.mark.asyncio
@patch("builtins.print")
async def test_arun_with_run_manager(
self,
mock_print,
search_tool,
mock_api_wrapper,
sample_raw_results,
sample_cleaned_results,
):
"""Test async run with callback manager."""
mock_run_manager = Mock()
mock_api_wrapper.raw_results_async = AsyncMock(return_value=sample_raw_results)
mock_api_wrapper.clean_results_with_images.return_value = sample_cleaned_results
result, raw = await search_tool._arun(
"test query", run_manager=mock_run_manager
)
assert result == sample_cleaned_results
assert raw == sample_raw_results
@patch("builtins.print")
def test_print_output_format(
self,
mock_print,
search_tool,
mock_api_wrapper,
sample_raw_results,
sample_cleaned_results,
):
"""Test that print outputs correctly formatted JSON."""
mock_api_wrapper.raw_results.return_value = sample_raw_results
mock_api_wrapper.clean_results_with_images.return_value = sample_cleaned_results
search_tool._run("test query")
# Verify print was called with expected format
call_args = mock_print.call_args[0]
assert call_args[0] == "sync"
assert isinstance(call_args[1], str) # Should be JSON string
# Verify it's valid JSON
json_data = json.loads(call_args[1])
assert json_data == sample_cleaned_results
@pytest.mark.asyncio
@patch("builtins.print")
async def test_async_print_output_format(
self,
mock_print,
search_tool,
mock_api_wrapper,
sample_raw_results,
sample_cleaned_results,
):
"""Test that async print outputs correctly formatted JSON."""
mock_api_wrapper.raw_results_async = AsyncMock(return_value=sample_raw_results)
mock_api_wrapper.clean_results_with_images.return_value = sample_cleaned_results
await search_tool._arun("test query")
# Verify print was called with expected format
call_args = mock_print.call_args[0]
assert call_args[0] == "async"
assert isinstance(call_args[1], str) # Should be JSON string
# Verify it's valid JSON
json_data = json.loads(call_args[1])
assert json_data == sample_cleaned_results

Some files were not shown because too many files have changed in this diff Show More