/** * Embedded browser SOP baseline — the zero-dependency fallback served by * {@link SopRegistry} when the `.workflow/knowhow/` directory has no overriding * browser SOP documents. Content is the verbatim former `BROWSER_SOPS` map * extracted from `browser-tool.ts`; knowhow entries with `tools: [browser]` + * `sop_topic` override these by the registry's merge rules. */ import type { EmbeddedSopMap } from "../sop-types.ts"; const SOP_CORE = `Browser SOP — when to use which tab.* helper and field-tested recipes. 1. MODE CHOICE (first decision) - Pure scraping (no login/CAPTCHA): open { visible:false } (headless default). - Login state / CAPTCHA / real fingerprint: open { visible:true, app:{ attach_user_profile:true, user_profile_dir } }. Pure stealth is NOT enough for Cloudflare managed challenges — attaching the user's real browser is the working path. - attach setup: pi auto-launches the user's Chrome with --remote-debugging-port=9222 --user-data-dir= when no live debug port is found, so just pass attach_user_profile + user_profile_dir. If a Chrome with that profile is already running with a debug port, pi reuses it. 2. CLOUDFLARE TURNSTILE (verified on NewAPI) - Attach the real browser (step 1) — CF trusts the real fingerprint. - Fetch the real sitekey: GET /api/status -> data.turnstile_site_key (backend-configured, not hardcoded). - Explicit render (render=explicit sites do NOT auto-render): const token = await tab.evaluate((sitekey) => new Promise((resolve) => { const c = document.createElement("div"); c.id="pi-ts"; c.style.cssText="position:fixed;top:60px;right:20px;z-index:99999;"; document.body.appendChild(c); let done=false, tok=""; const fin=(r)=>{ if(!done){done=true; resolve(r);} }; window.turnstile.render("#pi-ts", { sitekey, callback:(t)=>{tok=t;}, "error-callback":(e)=>fin({ok:false,error:"error:"+e}), "timeout-callback":()=>fin({ok:false,error:"timeout"}) }); let w=0; const iv=setInterval(()=>{ w+=500; if(tok){clearInterval(iv);fin({ok:true,token:tok,waited:w});} else if(w>=20000){clearInterval(iv);fin({ok:false,error:"no-token",waited:w});} }, 500); }), sitekey); - Token transport varies per site: NewAPI sends it as URL query param (?turnstile=...), NOT a body field. Reverse-engineer: search the JS bundle for /api/user/register and check params:{turnstile:...}. - Token is one-shot, ~5min lifetime — render + submit inside one run. - Pitfalls: isolated launch profile -> checkbox bounces back / infinite "verifying"; re-goto before each render. 3. CAPABILITY -> HELPER - Raw CDP domain: tab.cdp(method, params) -> raw JSON. High-risk: Page.crash / Browser.close terminate the session. - Cookies: tab.cookies.get/set/delete (session-level; in attach mode user login cookies are present; HttpOnly set needs httpOnly:true). - File upload: tab.uploadFile(selector, ...paths) (paths relative to cwd); transient input -> tab.cdp('DOM.setFileInputFiles', ...). - Cross-origin iframe: tab.evalInFrame(matcher, fn, ...args) (matcher = url substring/RegExp/predicate). - Open Shadow DOM: tab.pierce(selector) -> {x,y}; follow with tab.cdpClick(x,y). - Closed Shadow DOM: no selector engine can cross it; fallback tab.cdp('DOM.getDocument',{depth:-1,pierce:true}) + DOM.querySelector stepwise (host first, then inside its shadow). - Physical-coord click: tab.cdpClick(x,y,{hoverMs?}) — CDP Input 3-event (moved->pressed->released); canvas/non-DOM/hover-dependent. - Autofill release: tab.autofillRelease(selector) — bringToFront + cdpClick + re-dispatch input/change (foreground tab only). - Download-dialog bypass: tab.setDownloadBehavior(dirPath). - Multi-CDP chain: tab.cdpBatch([{method,params},...]) with "$N.path" refs (0-indexed); check each result.ok. - On-page OCR / visual localization: tab.ocr({region?,langs?}) -> {text, lines:[{bbox,text,confidence}]}; tab.detect({mode?,langs?}) -> {items:[{bbox,type,label,confidence}]} for canvas/non-DOM buttons. Follow with tab.cdpClick(cx, cy). Default langs is "eng" (pass "eng+chi_sim" for Chinese). Uses the shared local RapidOCR/OmniParser service and manifest-listed model assets; missing or unverified assets return {ok:false,error,hint,engine} and detection fails closed (no fabricated icons). For text-only needs without local models, describe_image can read text but cannot return reliable pixel coordinates. 4. CDP COORDINATE PITFALLS (field-tested) - Never skip mouseMoved: hover-dependent components (MUI Tooltip, Ant Dropdown) won't open without a hover dwell. - First-attach infobar offset: Chrome shows a ~20px "automated control" infobar on first CDP attach. If you measure coords before attach then click after, coords shift. Fix: send a harmless mouseMoved(0,0) first to stabilize. - Iframe targets: add iframe offset, finalX = iframeRect.x + elRect.x. - transform:scale/zoom: realX = x * zoom (zoom = parseFloat(getComputedStyle(document.documentElement).zoom) || 1). 5. FILE UPLOAD FALLBACK (isTrusted) puppeteer uploadFile does not fire isTrusted events; some frameworks don't notice. DataTransfer API fallback (pure JS): const file = new File([content], name, { type: "application/pdf" }); const dt = new DataTransfer(); dt.items.add(file); input.files = dt.files; input.dispatchEvent(new Event("input", { bubbles: true })); input.dispatchEvent(new Event("change", { bubbles: true })); 6. NAVIGATION SPLIT location.href nav + then operate in the SAME run -> "Inspected target navigated or closed" (context destroyed). Split into two runs: tab.goto -> wait -> separate run for operations. 7. CONNECT TROUBLESHOOTING - Browser not running? open a normal URL (about:blank does not load extensions / turnstile script). - Debug port not listening? pi auto-launches the user's Chrome with --remote-debugging-port=9222 on attach; if launch fails (profile locked, executable not found), set app.path / PUPPETEER_EXECUTABLE_PATH / CHROME_PATH, or start Chrome manually with --remote-debugging-port=9222 --user-data-dir= and retry. - attach error? if pi launched Chrome but no DevToolsActivePort appeared within ~15s, the profile may be locked by another Chrome instance — close it and retry. The auto-launched Chrome is detached and stays alive after pi exits; a later attach reuses the live port. `; const SOP_NETWORK = `Network interception & mocking — full access to requests/responses without a proxy server. REQUEST-LEVEL (puppeteer native, inside run code) - await page.setRequestInterception(true); page.on('request', (req) => { if (/analytics|ads|fonts/.test(req.url())) req.abort(); else req.continue(); }); - Blocking third-party noise speeds up loads and reduces detection surface. - Mock an API fixture: req.respond({ status: 200, contentType: 'application/json', body: JSON.stringify(data) }) — stabler than clicking through UI to reach state. RESPONSE-BODY REWRITE (puppeteer cannot do this natively; use a CDP session) - const s = await page.createCDPSession(); - await s.send('Fetch.enable', { patterns: [{ urlPattern: '**/api/*', requestStage: 'Response' }] }); - s.on('Fetch.requestPaused', async (e) => { const r = await s.send('Fetch.getResponseBody', { requestId: e.requestId }); /* modify */ await s.send('Fetch.fulfillRequest', { requestId: e.requestId, responseCode: e.responseStatusCode ?? 200, body: newBody }); }); - PITFALL: every matching request pauses while Fetch is enabled — fulfill/fail/continue EACH paused event or the page hangs forever. - Capture-only alternative: page.on('response') + response.json()/text() to harvest XHR payloads (batch APIs, tokens) without touching traffic. `; const SOP_AUTH = `Login, OAuth & verification-code flows. SESSION REUSE FIRST - The cheapest login is the one you skip: in attach mode the user's cookies are already present — probe an authed URL and confirm login state BEFORE driving any credential form. - After a programmatic login, export cookies (tab.cookies.get) so later runs can restore the session instead of re-logging-in. CREDENTIAL FORMS - Password fields: prefer real key events (tab.type/keyboard) over value injection — some frameworks bind on keydown and ignore synthetic input. TOTP 2FA - Holding the TOTP secret? Generate codes locally (RFC 6238, e.g. npm otplib) and fill the code input — no phone needed. Generate right before typing; if the 30s window has <2s left, wait for rollover first. EMAIL/SMS OTP - Flow: trigger send -> poll the inbox via API (IMAP or provider REST) -> extract the code with a contextual regex (near "code"/"verification code", usually 4-8 digits) -> type it. Poll with backoff up to ~60s; codes are single-use and expire in ~5-10min. OAUTH POPUPS - Consent screens often open a popup/new target: detect via run-output newTabs or tab.tabs(), drive THAT tab, then return to the opener. Do not launch with popup-blocking flags. POST-LOGIN ASSERTION - Interstitials ("checking browser", device-verification prompts) sit between submit and success: assert a logged-in marker (avatar element, account URL, cookie name) before continuing — see automation-antipatterns. `; const SOP_WIDGETS = `Complex form controls — custom widgets that resist fill()/click(). CONTENTEDITABLE RICH TEXT (ProseMirror/Slate/Quill/Jodit/Lark editor) - Focus the editor BEFORE setting text: fill()/value writes APPEND instead of replace when the element is not focused. - These are not : "Element is not an input" means click into the editor, then type with real key events; for structured content dispatch paste events with a text/html payload. CUSTOM DROPDOWNS / COMBOBOXES (antd Select, react-select, typeahead) - There are no native