fix(web_fetch): support proxy for Jina reader in restricted networks (#3418) (#3430)

* fix(web_fetch): support proxy for Jina reader in restricted networks

The web_fetch tool built a bare httpx.AsyncClient() with no proxy
awareness, so users behind a corporate proxy / in Docker / WSL could
not reach https://r.jina.ai and web_fetch timed out.

- Add optional `proxy` / `trust_env` params to JinaClient.crawl and
  wire them from the `web_fetch` tool config (with type coercion for
  YAML string values).
- Pass internal service hostnames through NO_PROXY in both compose
  files so proxy env inherited via env_file does not break in-cluster
  calls (gateway/provisioner/etc).
- Load proxy vars from .env into the shell in scripts/docker.sh so the
  NO_PROXY interpolation can merge user-provided values on `make` path.
- Document proxy/trust_env options in config.example.yaml.

Closes #3418

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Ryker_Feng
2026-06-08 23:25:29 +08:00
committed by GitHub
parent 3b6dd0a4e3
commit f92a26d56f
7 changed files with 254 additions and 6 deletions
+4
View File
@@ -537,6 +537,10 @@ tools:
group: web
use: deerflow.community.jina_ai.tools:web_fetch_tool
timeout: 10
# Optional proxy for restricted networks / Docker / WSL.
# Use host.docker.internal instead of 127.0.0.1 when the proxy runs on the host.
# proxy: $HTTPS_PROXY
# trust_env: true
# Web fetch tool (uses InfoQuest)
# - name: web_fetch