feat: support connecting to ws directly

[danamajid]

This adds support for using any ws CDP based server to connect to, without breaking existing usage of the browser rendering api. Example usage with Browserbase:

import { Browserbase } from '@browserbasehq/sdk';
import { chromium } from '@cloudflare/playwright';

const bb = new Browserbase({ apiKey: env.BROWSERBASE_API_KEY });
const session = await bb.sessions.create({
  projectId: env.BROWSERBASE_PROJECT_ID,
});

const browser = await chromium.connectOverCDP(session.connectUrl);

const defaultContext = browser.contexts()[0];
const page = defaultContext.pages()[0];

await page.goto('https://google.com');
page.close();
browser.close();

[danamajid]

As incoming CDP messages may be larger than the 1mb workerd allows (for example on websites with huge DOM), users may run into WebSocket protocol error; protocolError.statusCode = 1009; protocolError.description = Message is too large: 2417523 > 1048576

Luckily cloudflare/workers-sdk#6424 introduced increase_websocket_message_size compatibility flag.

Edit: nvm this flag can't be deployed to production atm.

[ruifigueira]

Yes, that would only work in dev, that's why we need chunking :( Eventually we could allow disabling chunks as you are proposing, but we would need to make it very clear that messages greater than 1MB would fail. And we still have the latency to take into account, because CDP is a very low level protocol, and for instance opening a new page can trigger 50 CDP calls.

[danamajid]

Yeah totally makes sense for it to chunk when communicating with Cloudflare's Browser Rendering API given you guys control the backend and have built support for it. But in order to support connecting to any arbitrary CDP server using connectOverCDP, this needs to be configurable. Apparently there's a compatibility flag called increase_websocket_message_size in Workers SDK that can be turned on to make this work, but this isn't yet available on production.

Unclear whether it'll never make it to production or just forgotten (since it got introduced for the debugger in dev mode originally) :) I've asked about this in cloudflare/workerd#4649 and also chatted with @threepointone. Would be a nice unlock for Cloudflare :)