feat(webfetch): route through fetch-use when BROWSER_USE_API_KEY is set

[Alezander9]

Summary

Routes the webfetch tool through Browser Use's fetch-use cloud (Chrome JA4 fingerprint, HTTP/2 header order, session-based cookie persistence, proxy pool) whenever BROWSER_USE_API_KEY is set. Falls back to the existing native HttpClient + cloudflare-retry path otherwise.

Implements ROADMAP B1 + B2. B3 retired by Phase H (Python harness goes away).

What changed

• New packages/bcode-browser/src/fetch-use.ts (~80 LOC): Effect service. Reads BROWSER_USE_API_KEY once at layer init, exposes enabled getter and fetch(url, opts) -> {body, contentType, statusCode} method. POSTs to https://fetch.browser-use.com/fetch with X-Browser-Use-API-Key.
• Modified packages/opencode/src/tool/webfetch.ts: one branch in execute swaps the request when fetchUse.enabled. Existing post-processing (image-attachment / markdown / text / html conversion) is unchanged. No schema changes.
• Modified packages/opencode/src/tool/registry.ts: adds FetchUse.Service to the layer requirements + Layer.provide(FetchUse.layer) to defaultLayer.
• Tests updated to provide FetchUse.layer where ToolRegistry/WebFetchTool is wired.

Why direct Level-3 mod, not a plugin hook or upstream PR

We own browsercode/. Reducing diff against anomalyco/opencode matters for upstream-pull friction, not approval. Adding plugin-hook indirection inside our fork would touch more files (webfetch.ts + plugin types + registration) for a single fetch backend. The direct branch is smallest-total-code.

Opt-out

BCODE_NO_FETCH_USE=1 env var. Chose env over opencode.json config because it keeps the diff bounded to one upstream file.

Schema deferred

Originally planned session_id / output_format / proxy_country schema additions on the webfetch tool are deferred. They add prompt verbosity for features evals havent shown we need. Easy to add later if demand emerges.

Verified

• bun typecheck clean across bcode-browser and opencode.
• packages/opencode: 3/3 webfetch tests pass (native fallback path unchanged).
• packages/bcode-browser/test/fetch-use.test.ts:
  • Unit: enabled reflects env vars correctly.
  • Live (against production fetch.browser-use.com): POST + decode round-trip against https://httpbin.org/get returned 200 with application/json and the expected body. Skipped when BROWSER_USE_API_KEY is unset.

Notes

packages/bcode-browser/harness/ is unrelated untracked content and not part of this PR.

[cubic-dev-ai]

3 issues found across 8 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/opencode/test/tool/webfetch.test.ts">

<violation number="1" location="packages/opencode/test/tool/webfetch.test.ts:36">
P2: Don’t enable FetchUse unconditionally in this test harness; it makes the suite environment-dependent and can break when BROWSER_USE_API_KEY is present.</violation>
</file>

<file name="packages/bcode-browser/test/fetch-use.test.ts">

<violation number="1" location="packages/bcode-browser/test/fetch-use.test.ts:26">
P2: Skip the live fetch-use test when the opt-out env var is set too, not just when the API key is missing.</violation>
</file>

<file name="packages/opencode/src/tool/webfetch.ts">

<violation number="1" location="packages/opencode/src/tool/webfetch.ts:59">
P1: The fetch-use branch ignores the fetched page status code, so HTTP error pages (e.g. 404/500) are treated as successful content instead of failing like the native path.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.}

[cubic-dev-ai]

P1: The fetch-use branch ignores the fetched page status code, so HTTP error pages (e.g. 404/500) are treated as successful content instead of failing like the native path.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/tool/webfetch.ts, line 59:

<comment>The fetch-use branch ignores the fetched page status code, so HTTP error pages (e.g. 404/500) are treated as successful content instead of failing like the native path.</comment>

<file context>
@@ -47,61 +49,77 @@ export const WebFetchTool = Tool.define(
+          const { arrayBuffer, contentType } = yield* (fetchUse.enabled
+            ? fetchUse
+                .fetch(params.url, { timeoutMs: timeout })
+                .pipe(Effect.map((r) => ({ arrayBuffer: r.body, contentType: r.contentType })))
+            : Effect.gen(function* () {
+                // Build Accept header based on requested format with q parameters for fallbacks
</file context>

Suggested change

	.pipe(Effect.map((r) => ({ arrayBuffer: r.body, contentType: r.contentType })))
	.pipe(
	Effect.flatMap((r) =>
	r.statusCode >= 200 && r.statusCode < 300
	? Effect.succeed({ arrayBuffer: r.body, contentType: r.contentType })
	: Effect.fail(new Error(`Request failed with status ${r.statusCode}`)),
	),
	)

[cubic-dev-ai]

P2: Skip the live fetch-use test when the opt-out env var is set too, not just when the API key is missing.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/bcode-browser/test/fetch-use.test.ts, line 26:

<comment>Skip the live fetch-use test when the opt-out env var is set too, not just when the API key is missing.</comment>

<file context>
@@ -0,0 +1,40 @@
+  expect(enabled).toBe(haveKey && process.env.BCODE_NO_FETCH_USE !== "1")
+})
+
+test.skipIf(!haveKey)("live: fetches httpbin and returns body + content-type", async () => {
+  const result = await Effect.gen(function* () {
+    const svc = yield* FetchUse.Service
</file context>