Files
deer-flow/backend/tests/test_features_router.py
fa496c0c8d feat(browser): add agentic browser control (#4187)
* feat(browser): add agentic browser control

* fix(frontend): format browser view changes

* fix(browser): keep browser optional and isolate sidecar layout

* fix(browser): address PR review security and IME findings

- Nginx: add a browser-stream WebSocket location before the generic
  /api/threads regex so Live upgrades instead of downgrading to HTTP
  (both nginx.conf and nginx.local.conf).
- Ownership: require an existing owned thread for the WS stream and REST
  navigate, and tear down the browser session on thread deletion so a
  later caller cannot reuse a retained page/cookies by guessing the id.
- SSRF: enforce the URL policy at the browser request boundary via a
  context-level route guard covering redirects, popups, iframes, and
  subresources (skipped for CDP-attached Chrome).
- IME: skip key forwarding while a composition is active so confirming a
  CJK candidate with Enter no longer submits the remote page form.

Adds regression tests for the request guard, session teardown on delete,
and the composing-Enter key decision.

* fix(frontend): smooth streaming in long tool threads

* Revert "fix(frontend): smooth streaming in long tool threads"

This reverts commit f0462516ea.

* fix(browser): address review security and lifecycle findings

- Reject cross-origin WebSocket upgrades on the live browser stream
  (Origin allow-list reuse of CORS/same-origin helpers) to close a
  WS-CSRF hole, and fail closed when the ownership store is absent.
- Warn when a CDP-attached session runs with the SSRF request guard
  off, and drop the unreachable CDP screencast teardown dead code.
- Read browser session launch config from a single canonical source
  (browser_navigate) so it is deterministic regardless of call order.
- Bound per-thread Chromium accumulation with idle-timeout eviction
  and an LRU max-sessions cap.
- Reset the Live reconnect counter on a successful open so the stream
  can't permanently stall after the cumulative attempt cap.

* fix(frontend): reduce long tool thread render stalls

Reuse stable historical message groups during streaming, defer heavy Markdown and browser previews, and lazy-decode message images.

* fix(browser): keep live control responsive during continuous input

Why: Manual browser control felt laggy — a physical click ran the remote
Playwright click three times and each non-move input synchronously awaited a
JPEG screenshot, so events queued behind capture (queue wait up to ~237ms).
The first async attempt used a trailing-edge debounce, which froze the visible
page until a wheel/keyboard gesture stopped ("scroll finishes, then it jumps").

What:
- Frontend forwards one `click` per physical click instead of also emitting
  `down`/`up`, so the remote page is not clicked twice per gesture.
- Backend detaches live-frame capture from input dispatch: non-move actions
  start a rate-limited background refresh loop (leading frame + bounded cadence)
  that keeps emitting frames while input continues and never blocks dispatch.
- Add regression tests: input dispatch no longer awaits the screenshot, rapid
  inputs coalesce, and continuous input keeps refreshing before it stops.

Scenarios: Verified in the live Browser panel — a single click completes in
~57ms (was blocked behind a 171ms capture), and a 1.14s sustained wheel gesture
renders ~7 frames throughout the scroll instead of one frame after it ends.

* fix(browser): harden worker and session lifecycle

* fix(browser): address latest review feedback

* fix(frontend): preserve optimistic new-chat message

* test(e2e): preserve mocked message run ids

* fix(browser): address capability review feedback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-21 11:46:33 +08:00

74 lines
2.8 KiB
Python

from types import SimpleNamespace
from unittest.mock import patch
from fastapi import FastAPI
from fastapi.testclient import TestClient
from app.gateway.deps import get_config
from app.gateway.routers import features
def _app_with_config(*, agents_api_enabled: bool, browser_enabled: bool = False, browser_extra: dict | None = None) -> FastAPI:
app = FastAPI()
app.include_router(features.router)
tools = (
[
SimpleNamespace(name="browser_navigate", model_extra=browser_extra or {}),
]
if browser_enabled
else []
)
fake_config = SimpleNamespace(agents_api=SimpleNamespace(enabled=agents_api_enabled), tools=tools)
app.dependency_overrides[get_config] = lambda: fake_config
return app
def test_features_reports_agents_api_enabled() -> None:
with TestClient(_app_with_config(agents_api_enabled=True)) as client:
response = client.get("/api/features")
assert response.status_code == 200
assert response.json() == {"agents_api": {"enabled": True}, "browser_control": {"enabled": False}}
def test_features_reports_agents_api_disabled() -> None:
with TestClient(_app_with_config(agents_api_enabled=False)) as client:
response = client.get("/api/features")
assert response.status_code == 200
assert response.json() == {"agents_api": {"enabled": False}, "browser_control": {"enabled": False}}
def test_features_reports_browser_control_enabled_when_configured_and_runtime_available() -> None:
with (
patch("app.gateway.browser_capability.importlib.util.find_spec", return_value=object()),
TestClient(_app_with_config(agents_api_enabled=True, browser_enabled=True)) as client,
):
response = client.get("/api/features")
assert response.status_code == 200
assert response.json()["browser_control"] == {"enabled": True}
def test_features_reports_browser_control_disabled_when_runtime_missing() -> None:
with (
patch("app.gateway.browser_capability.importlib.util.find_spec", return_value=None),
TestClient(_app_with_config(agents_api_enabled=True, browser_enabled=True)) as client,
):
response = client.get("/api/features")
assert response.status_code == 200
assert response.json()["browser_control"] == {"enabled": False}
def test_features_reports_browser_control_disabled_for_unguarded_cdp() -> None:
with (
patch("app.gateway.browser_capability.importlib.util.find_spec", return_value=object()),
TestClient(
_app_with_config(
agents_api_enabled=True,
browser_enabled=True,
browser_extra={"cdp_url": "http://127.0.0.1:9222"},
),
) as client,
):
response = client.get("/api/features")
assert response.status_code == 200
assert response.json()["browser_control"] == {"enabled": False}