[ { "pathname": "/", "data": { "markdown": "Welcome to Hyperbrowser \\| Hyperbrowser\n\n![Page cover image](https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4095930873-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FfwZVRs9Jmwzw9cfbchYG%252Fuploads%252FMZLDMBau66kOiuK87ulA%252Fbrowser.webp%3Falt%3Dmedia%26token%3D49ad4828-4865-427d-8262-691490e1c52d&width=1248&dpr=4&quality=100&sign=165a6d2f&sv=2)\n\nWelcome to Hyperbrowser, the Internet for AI. Hyperbrowser is the next-generation platform empowering AI agents and enabling effortless, scalable browser automation. Built specifically for AI developers, it eliminates the headaches of local infrastructure and performance bottlenecks, allowing you to focus entirely on building your solutions, rather getting gummed up on browser problems.\n\nWhether you're training AI agents to navigate the web, collecting data for model fine-tuning, testing applications, or simply scraping data, Hyperbrowser lets you launch and manage browser sessions with ease鈥攏o complicated setup required. Our platform provides streamlined solutions for all your web scraping needs, from single-page extraction to comprehensive site crawling.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/\\#why-developers-choose-hyperbrowser) Why Developers Choose Hyperbrowser:\n\n- **Instant Scalability** \\- Deploy hundreds of AI agent browser sessions in seconds without infrastructure complexity\n\n- **Powerful APIs** \\- Purpose-built APIs for managing sessions, training environments, scraping/crawling sites, and enhancing AI capabilities\n\n- **Production-Ready AI Infrastructure** \\- Enterprise-grade reliability and security built specifically for AI workloads\n\n- **Advanced Anti-Bot Protection Bypass** \\- Built-in stealth mode, ad blocking, automatic CAPTCHA solving, and rotating proxies for uninterrupted AI operation\n\n- **AI-First Design** \\- Native support for multiple AI frameworks including LangChain, LlamaIndex, MCP, and more\n\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/\\#quick-example) Quick Example\n\nStart automating in just a few lines of code\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { connect } from \"puppeteer-core\";\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst session = await client.sessions.create();\n\n// Use the session to automate browser actions\nconst browser = await connect({\n browserWSEndpoint: session.wsEndpoint,\n defaultViewport: null,\n});\n\n// Use the browser to automate browser actions\nconst page = await browser.newPage();\nawait page.goto(\"https://example.com\");\n\nawait browser.close();\n\n// Once done, you can stop the session\nawait client.sessions.stop(session.id);\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/\\#jump-right-in) Jump right in\n\n[馃摉\\\\\n\\\\\n**Scraping**\\\\\n\\\\\nScrape a site and get its contents in markdown](https://www.docs.hyperbrowser.ai/get-started/quickstart/scraping) [馃懆鈥嶐煉籠\\\\n\\\\\n**Puppeteer**\\\\\n\\\\\nConnect to a browser session with Puppeteer](https://www.docs.hyperbrowser.ai/get-started/quickstart/puppeteer) [馃暦锔廫\\\\n\\\\\n**Crawling**\\\\\n\\\\\nCrawl an entire site and all its linked pages](https://www.docs.hyperbrowser.ai/get-started/quickstart/crawling) [馃椉\\\\\n\\\\\nStructured Extraction\\\\\n\\\\\nPerform extraction into a json schema](https://www.docs.hyperbrowser.ai/web-scraping/extract)\n\n[NextWhat are Headless browsers ?](https://www.docs.hyperbrowser.ai/what-are-headless-browsers)\n\nLast updated 7 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/", "title": "Welcome to Hyperbrowser | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/kzTlst3tKo255yz4YpDi", "og:title": "Welcome to Hyperbrowser | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (f80c3a7)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/kzTlst3tKo255yz4YpDi", "twitter:title": "Welcome to Hyperbrowser | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai", "status": "completed", "error": null }, "summary": "Hyperbrowser documentation provides an introduction to web scraping and automation using the Hyperbrowser tool." }, { "pathname": "/what-are-headless-browsers", "data": { "markdown": "What are Headless browsers ? \\| Hyperbrowser\n\nIn the world of AI development, you'd most likely have come across the the term \"headless browser\" in tutorials and documentation. While it may sound technical and intimidating, the concept is straightforward and has been changing how AI interacts with the web.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#the-basics) The Basics\n\nA headless browser can be defined as a web browser without a graphical user interface鈥攐r \"head.\" Similar to Google Chrome or Firefox, but without any windows, buttons, or visual elements for human interaction.\n\nInstead of being controlled through clicking, scrolling, and typing manually, a headless browser is operated entirely through code. All standard browser functions are still performed:\n\n- Web pages are loaded\n\n- JavaScript is executed\n\n- Cookies and sessions are handled\n\n- Content is rendered\n\n- Forms are processed\n\n\nThe key distinction is that these actions occur invisibly, behind the scenes, directed by code rather than human intervention.\n\nSome advanced headless browsers may also render all the graphics that a normal browser does!\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#how-headless-browsers-function) How Headless Browsers Function\n\nWhen a headless browser is utilized, what would normally be done by a human in a regular browser is automated:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n// Example using Puppeteer with Hyperbrowser\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { connect } from \"puppeteer-core\";\n\n// A session is created\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\nconst session = await client.sessions.create();\n\n// Connection to the browser is established\nconst browser = await connect({\n browserWSEndpoint: session.wsEndpoint,\n defaultViewport: null,\n});\n\n// Navigation to a website is performed\nconst page = await browser.newPage();\nawait page.goto(\"https://www.search-example.com\");\n\n// Elements are found and interactions are executed\nawait page.type(\"textarea\", \"AI development\");\nawait page.click(\"button#Search\");\n\n// Results are awaited and data is extracted\nawait page.waitForSelector(\".results\");\nconst data = await page.evaluate(() => {\n return Array.from(document.querySelectorAll(\".result-item\"))\n .map(item => item.textContent);\n});\n\n// Resources are released\nawait browser.close();\nawait client.sessions.stop(session.id);\n```\n\nIn this example, the following actions are programmatically executed:\n\n1. A browser is opened\n\n2. A website is navigated to\n\n3. Text is entered into a search box\n\n4. A button is clicked\n\n5. Results are waited for\n\n6. Data is extracted from those results\n\n\nAll of these processes are completed without a browser window being displayed.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#advantages-of-headless-browsers-over-regular-browsers) Advantages of Headless Browsers Over Regular Browsers\n\nThe advantages of headless browsers over traditional browsers become apparent when implementation is considered:\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#resource-efficiency) Resource Efficiency\n\nMemory and CPU consumption are significantly reduced with headless browsers since visual rendering is not required. This efficiency allows for many more instances to be run simultaneously.\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#speed) Speed\n\nTasks can be executed much faster without the overhead of rendering visual elements for human consumption.\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#scalability) Scalability\n\nWhen hundreds of web pages need to be processed simultaneously, headless browsers can be scaled up much more effectively than their headed counterparts.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#challenges-in-headless-browser-implementation) Challenges in Headless Browser Implementation\n\nSeveral challenges may be encountered when headless browsers are implemented independently:\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#infrastructure-complexity) Infrastructure Complexity\n\nSignificant technical expertise and infrastructure are required to manage browser instances, especially at scale.\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#anti-bot-detection) Anti-Bot Detection\n\nAutomated browsing is often detected and blocked by many websites, which creates difficulties in data gathering or functionality testing.\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#performance-limitations) Performance Limitations\n\nAvailable resources are quickly consumed when multiple browser instances are run locally, which limits operational capacity.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/what-are-headless-browsers\\#cloud-based-solutions-such-as-hyperbrowser) Cloud-Based Solutions Such as Hyperbrowser\n\nServices like Hyperbrowser are designed to address these challenges. Instead of local setup and limitations being struggled with, Hyperbrowser provides:\n\n- **Instant Scalability**: Hundreds of browser sessions can be initiated without infrastructure complications\n\n- **Built-in Anti-Bot Countermeasures**: Automation can be kept running smoothly with features like stealth mode and CAPTCHA solving\n\n- **Simple Integration**: Familiar tools like Puppeteer and Playwright are supported\n\n- **API-First Design**: Sessions can be managed and data can be extracted with easy-to-use APIs\n\n\n[PreviousWelcome to Hyperbrowser](https://www.docs.hyperbrowser.ai/) [NextQuickstart](https://www.docs.hyperbrowser.ai/get-started/quickstart)\n\nLast updated 8 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/what-are-headless-browsers", "title": "What are Headless browsers ? | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/0vtWGIlGE4NbAu59RGTs", "og:title": "What are Headless browsers ? | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/what-are-headless-browsers", "description": "And more importantly, why should I care ?", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/0vtWGIlGE4NbAu59RGTs", "twitter:title": "What are Headless browsers ? | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/what-are-headless-browsers", "og:description": "And more importantly, why should I care ?", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "And more importantly, why should I care ?" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/what-are-headless-browsers", "status": "completed", "error": null }, "summary": "The page explains headless browsers and their role in Hyperbrowser for web scraping and automation tasks." }, { "pathname": "/get-started/quickstart/scraping", "data": { "markdown": "Scraping \\| Hyperbrowser\n\n1\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/scraping\\#install-hyperbrowser) Install Hyperbrowser\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install hyperbrowser\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nuv add hyperbrowse\n```\n\n2\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/scraping\\#setup-your-environment) Setup your Environment\n\nTo use Hyperbrowser with your code, you will need an API Key. You can get one easily from the [dashboard](https://app.hyperbrowser.ai/settings?tab=api_key). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY` .\n\n3\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/scraping\\#scrape-a-site) Scrape a Site\n\nNext, you can scrape any site by simply setting up the Hyperbrowser client and providing the site's url.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const scrapeResult = await client.scrape.startAndWait({\n url: \"https://example.com\",\n });\n console.log(\"Scrape result:\", scrapeResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser as Hyperbrowser\nfrom hyperbrowser.models.scrape import StartScrapeJobParams\n\n# Load environment variables from .env file\nload_dotenv()\n\n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n # Start scraping and wait for completion\n scrape_result = await client.scrape.start_and_wait(\n StartScrapeJobParams(url=\"https://example.com\")\n )\n print(\"Scrape result:\", scrape_result)\n\nif __name__ == \"__main__\":\n import asyncio\n\n asyncio.run(main())\n```\n\n4\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/scraping\\#view-session-in-dashboard) View Session in Dashboard\n\nYou can view all your sessions in the [dashboard](https://app.hyperbrowser.ai/) and see their recordings or other key metrics like logs.\n\nMore information about sessions, including user profiles, web recordings, live view, and more can be found in the [Sessions Overview](https://www.docs.hyperbrowser.ai/sessions/overview)\n\n[PreviousQuickstart](https://www.docs.hyperbrowser.ai/get-started/quickstart) [NextCrawling](https://www.docs.hyperbrowser.ai/get-started/quickstart/crawling)\n\nLast updated 8 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/get-started/quickstart/scraping", "title": "Scraping | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/IIcaSJSVwXpK0SQVg5vz", "og:title": "Scraping | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/get-started/quickstart/scraping", "description": "Scrape any site and get it's data.", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/IIcaSJSVwXpK0SQVg5vz", "twitter:title": "Scraping | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/get-started/quickstart/scraping", "og:description": "Scrape any site and get it's data.", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Scrape any site and get it's data." }, "html": null, "url": "https://www.docs.hyperbrowser.ai/get-started/quickstart/scraping", "status": "completed", "error": null }, "summary": "The \"Scraping\" page in Hyperbrowser details how to extract data from websites using the tool's functionalities." }, { "pathname": "/get-started/quickstart/crawling", "data": { "markdown": "Crawling \\| Hyperbrowser\n\n1\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/crawling\\#install-hyperbrowser) Install Hyperbrowser\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install hyperbrowser\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nuv add hyperbrowser\n```\n\n2\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/crawling\\#setup-your-environment) Setup your Environment\n\nTo use Hyperbrowser with your code, you will need an API Key. You can get one easily from the [dashboard](https://app.hyperbrowser.ai/settings?tab=api_key). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY` .\n\n3\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/crawling\\#crawl-a-site) Crawl a Site\n\nNext, you can crawl any site by simply setting up the Hyperbrowser client and providing the site's url.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const crawlResult = await client.crawl.startAndWait({\n url: \"https://hyperbrowser.ai\",\n });\n console.log(\"Crawl result:\", crawlResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser as Hyperbrowser\nfrom hyperbrowser.models.crawl import StartCrawlJobParams\n\n# Load environment variables from .env file\nload_dotenv()\n\n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n # Start crawling and wait for completion\n crawl_result = await client.crawl.start_and_wait(\n StartCrawlJobParams(url=\"https://hyperbrowser.ai\")\n )\n print(\"Crawl result:\")\n print(crawl_result.model_dump_json(indent=2))\n\nif __name__ == \"__main__\":\n import asyncio\n\n asyncio.run(main())\n```\n\n4\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/crawling\\#view-session-in-dashboard) View Session in Dashboard\n\nYou can view all your sessions in the [dashboard](https://app.hyperbrowser.ai/) and see their recordings or other key metrics like logs.\n\n[PreviousScraping](https://www.docs.hyperbrowser.ai/get-started/quickstart/scraping) [NextPuppeteer](https://www.docs.hyperbrowser.ai/get-started/quickstart/puppeteer)\n\nLast updated 8 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/get-started/quickstart/crawling", "title": "Crawling | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/igycZQMZI66cnXvNxbMh", "og:title": "Crawling | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/get-started/quickstart/crawling", "description": "Crawl a site and all it's links.", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/igycZQMZI66cnXvNxbMh", "twitter:title": "Crawling | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/get-started/quickstart/crawling", "og:description": "Crawl a site and all it's links.", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Crawl a site and all it's links." }, "html": null, "url": "https://www.docs.hyperbrowser.ai/get-started/quickstart/crawling", "status": "completed", "error": null }, "summary": "The \"Crawling\" page of Hyperbrowser covers the tool's web scraping capabilities and how to implement them." }, { "pathname": "/get-started/quickstart", "data": { "markdown": "Quickstart \\| Hyperbrowser\n\n![](https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4095930873-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FfwZVRs9Jmwzw9cfbchYG%252Fuploads%252FVjV04cIkThJDJ36JJhPF%252FUntitled.png%3Falt%3Dmedia%26token%3D0f3c2955-2d73-4b0f-8adc-428eeb96ca02&width=768&dpr=4&quality=100&sign=510f4f50&sv=2)\n\nWelcome to Hyperbrowser, the Internet for AI. Get started in minutes with our browser automation, scraping, and crawling tools.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart\\#get-started-in-three-simple-steps) Get Started in Three Simple Steps\n\n1. **Create an Account**: Sign up at the [dashboard](https://app.hyperbrowser.ai/signup)\n\n2. **Get Your API Key**: Find it in your [account settings](https://app.hyperbrowser.ai/settings?tab=api_key)\n\n3. **Write your first scrape**\n\n\nNodeJSPython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst scrapeResult = await client.scrape.startAndWait({\n url: \"https://example.com\",\n});\nconsole.log(scrapeResult.content);\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nfrom hyperbrowser import AsyncHyperbrowser\nfrom hyperbrowser.models.scrape import StartScrapeJobParams\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nscrape_result = await client.scrape.start_and_wait(\n StartScrapeJobParams(url=\"https://example.com\")\n)\nprint(scrape_result.content)\n```\n\nCongratulations, you've scraped your first web page ! Welcome to the world of AI agents, and browser automations\n\nCheck out our in-depth SDK references for [Node](https://www.docs.hyperbrowser.ai/reference/sdks/node) and [Python](https://www.docs.hyperbrowser.ai/reference/sdks/python).\n\n[PreviousWhat are Headless browsers ?](https://www.docs.hyperbrowser.ai/what-are-headless-browsers) [NextScraping](https://www.docs.hyperbrowser.ai/get-started/quickstart/scraping)\n\nLast updated 8 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/get-started/quickstart", "title": "Quickstart | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/CyH2xJQs9yWJ1S8BYNav", "og:title": "Quickstart | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/get-started/quickstart", "description": "Get setup with Hyperbrowser in minutes.", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/CyH2xJQs9yWJ1S8BYNav", "twitter:title": "Quickstart | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/get-started/quickstart", "og:description": "Get setup with Hyperbrowser in minutes.", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Get setup with Hyperbrowser in minutes." }, "html": null, "url": "https://www.docs.hyperbrowser.ai/get-started/quickstart", "status": "completed", "error": null }, "summary": "Quickstart guide for Hyperbrowser provides initial setup and functionality instructions for effective web scraping and automation." }, { "pathname": "/get-started/quickstart/puppeteer", "data": { "markdown": "Puppeteer \\| Hyperbrowser\n\n1\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/puppeteer\\#install-puppeteer-and-hyperbrowser) Install Puppeteer and Hyperbrowser\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install puppeteer-core @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add puppeteer-core @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install pyppeteer hyperbrowser\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nuv add pyppeteer hyperbrowser\n```\n\n2\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/puppeteer\\#setup-your-environment) Setup your Environment\n\nTo use Hyperbrowser with your code, you will need an API Key. You can get one easily from the [dashboard](https://app.hyperbrowser.ai/settings?tab=api_key). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY` .\n\n3\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/puppeteer\\#setup-browser-session) Setup Browser Session\n\nNext, you can easily startup a browser session using puppeteer and your Hyperbrowser API Key.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { connect } from \"puppeteer-core\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst main = async () => {\n const browser = await connect({\n browserWSEndpoint: `wss://connect.hyperbrowser.ai?apiKey=${process.env.HYPERBROWSER_API_KEY}`,\n });\n\n const [page] = await browser.pages();\n\n // Navigate to a website\n console.log(\"Navigating to Hacker News...\");\n await page.goto(\"https://news.ycombinator.com/\");\n const pageTitle = await page.title();\n console.log(\"Page 1:\", pageTitle);\n await page.evaluate(() => {\n console.log(\"Page 1:\", document.title);\n });\n\n await page.goto(\"https://example.com\");\n console.log(\"Page 2:\", await page.title());\n await page.evaluate(() => {\n console.log(\"Page 2:\", document.title);\n });\n\n await page.goto(\"https://apple.com\");\n console.log(\"Page 3:\", await page.title());\n await page.evaluate(() => {\n console.log(\"Page 3:\", document.title);\n });\n\n await page.goto(\"https://google.com\");\n console.log(\"Page 4:\", await page.title());\n await page.evaluate(() => {\n console.log(\"Page 4:\", document.title);\n });\n\n // Clean up\n await browser.close();\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nfrom pyppeteer import connect\nimport os\nfrom dotenv import load_dotenv\n\n# Load environment variables from .env file\nload_dotenv()\n\nasync def main():\n # Connect to the browser using the API key from environment variables\n browser = await connect(\n browserWSEndpoint=f\"wss://connect.hyperbrowser.ai?apiKey={os.getenv('HYPERBROWSER_API_KEY')}\"\n )\n\n pages = await browser.pages()\n page = pages[0]\n\n # Navigate to a website\n print(\"Navigating to Hacker News...\")\n await page.goto(\"https://news.ycombinator.com/\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n await page.evaluate(\"() => { console.log('Page 1:', document.title); }\")\n\n await page.goto(\"https://example.com\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n await page.evaluate(\"() => { console.log('Page 2:', document.title); }\")\n\n await page.goto(\"https://apple.com\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n await page.evaluate(\"() => { console.log('Page 3:', document.title); }\")\n\n await page.goto(\"https://google.com\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n await page.evaluate(\"() => { console.log('Page 4:', document.title); }\")\n\n # Clean up\n await browser.close()\n\n# Run the async main function\nif __name__ == \"__main__\":\n asyncio.run(main())\n```\n\n4\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/puppeteer\\#view-session-in-dashboard) View Session in Dashboard\n\nYou can view all your sessions in the [dashboard](https://app.hyperbrowser.ai/) and see their recordings or other key metrics like logs.\n\n[PreviousCrawling](https://www.docs.hyperbrowser.ai/get-started/quickstart/crawling) [NextPlaywright](https://www.docs.hyperbrowser.ai/get-started/quickstart/playwright)\n\nLast updated 8 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/get-started/quickstart/puppeteer", "title": "Puppeteer | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/GWpUH7WQoSoNn5xG3QO6", "og:title": "Puppeteer | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (f80c3a7)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/get-started/quickstart/puppeteer", "description": "Setup a browser session with Puppeteer.", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/GWpUH7WQoSoNn5xG3QO6", "twitter:title": "Puppeteer | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/get-started/quickstart/puppeteer", "og:description": "Setup a browser session with Puppeteer.", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Setup a browser session with Puppeteer." }, "html": null, "url": "https://www.docs.hyperbrowser.ai/get-started/quickstart/puppeteer", "status": "completed", "error": null }, "summary": "Puppeteer integration with Hyperbrowser enables web scraping and automation through headless browser control." }, { "pathname": "/get-started/quickstart/playwright", "data": { "markdown": "Playwright \\| Hyperbrowser\n\n1\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/playwright\\#install-playwright-and-hyperbrowser) Install Playwright and Hyperbrowser\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install playwright-core @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add playwright-core @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install playwright hyperbrowser\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nuv add playwright hyperbrowser\n```\n\n2\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/playwright\\#setup-your-environment) Setup your Environment\n\nTo use Hyperbrowser with your code, you will need an API Key. You can get one easily from the [dashboard](https://app.hyperbrowser.ai/settings?tab=api_key). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY` .\n\n3\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/playwright\\#setup-browser-session) Setup Browser Session\n\nNext, you can easily startup a browser session using playwright and your Hyperbrowser API Key.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { chromium } from \"playwright-core\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst main = async () => {\n // Connect to browser using Playwright\n const browser = await chromium.connectOverCDP(\n `wss://connect.hyperbrowser.ai?apiKey=${process.env.HYPERBROWSER_API_KEY}`\n );\n\n // Create a new context and page\n const defaultContext = browser.contexts()[0];\n const page = defaultContext.pages()[0];\n\n // Navigate to a website\n console.log(\"Navigating to Hacker News...\");\n await page.goto(\"https://news.ycombinator.com/\");\n const pageTitle = await page.title();\n console.log(\"Page 1:\", pageTitle);\n await page.evaluate(() => {\n console.log(\"Page 1:\", document.title);\n });\n\n await page.goto(\"https://example.com\");\n console.log(\"Page 2:\", await page.title());\n await page.evaluate(() => {\n console.log(\"Page 2:\", document.title);\n });\n\n await page.goto(\"https://apple.com\");\n console.log(\"Page 3:\", await page.title());\n await page.evaluate(() => {\n console.log(\"Page 3:\", document.title);\n });\n\n await page.goto(\"https://google.com\");\n console.log(\"Page 4:\", await page.title());\n await page.evaluate(() => {\n console.log(\"Page 4:\", document.title);\n });\n\n // Clean up\n await defaultContext.close();\n await browser.close();\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom playwright.sync_api import sync_playwright\nfrom dotenv import load_dotenv\n\n# Load environment variables from .env file\nload_dotenv()\n\ndef main():\n with sync_playwright() as p:\n # Connect to browser using Playwright\n browser = p.chromium.connect_over_cdp(\n f\"wss://connect.hyperbrowser.ai?apiKey={os.getenv('HYPERBROWSER_API_KEY')}\"\n )\n\n # Get the default context and page\n default_context = browser.contexts[0]\n page = default_context.pages[0]\n\n # Navigate to various websites\n print(\"Navigating to Hacker News...\")\n page.goto(\"https://news.ycombinator.com/\")\n page_title = page.title()\n print(\"Page 1:\", page_title)\n page.evaluate(\"() => { console.log('Page 1:', document.title); }\")\n\n page.goto(\"https://example.com\")\n print(\"Page 2:\", page.title())\n page.evaluate(\"() => { console.log('Page 2:', document.title); }\")\n\n page.goto(\"https://apple.com\")\n print(\"Page 3:\", page.title())\n page.evaluate(\"() => { console.log('Page 3:', document.title); }\")\n\n page.goto(\"https://google.com\")\n print(\"Page 4:\", page.title())\n page.evaluate(\"() => { console.log('Page 4:', document.title); }\")\n\n # Clean up\n default_context.close()\n browser.close()\n\nif __name__ == \"__main__\":\n main()\n```\n\n4\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/playwright\\#view-session-in-dashboard) View Session in Dashboard\n\nYou can view all your sessions in the [dashboard](https://app.hyperbrowser.ai/) and see their recordings or other key metrics like logs.\n\n[PreviousPuppeteer](https://www.docs.hyperbrowser.ai/get-started/quickstart/puppeteer) [NextSelenium](https://www.docs.hyperbrowser.ai/get-started/quickstart/selenium)\n\nLast updated 8 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/get-started/quickstart/playwright", "title": "Playwright | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/LmAydj7A6NczbKuRAunj", "og:title": "Playwright | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/get-started/quickstart/playwright", "description": "Setup a browser session with Playwright.", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/LmAydj7A6NczbKuRAunj", "twitter:title": "Playwright | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/get-started/quickstart/playwright", "og:description": "Setup a browser session with Playwright.", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Setup a browser session with Playwright." }, "html": null, "url": "https://www.docs.hyperbrowser.ai/get-started/quickstart/playwright", "status": "completed", "error": null }, "summary": "The page discusses using Playwright with Hyperbrowser for web scraping and automation tasks." }, { "pathname": "/get-started/quickstart/selenium", "data": { "markdown": "Selenium \\| Hyperbrowser\n\n1\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/selenium\\#install-selenium-and-hyperbrowser) Install Selenium and Hyperbrowser\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install selenium-webdriver @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add selenium-webdriver @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install selenium hyperbrowser\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nuv add selenium hyperbrowser\n```\n\n2\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/selenium\\#setup-your-environment) Setup your Environment\n\nTo use Hyperbrowser with your code, you will need an API Key. You can get one easily from the [dashboard](https://app.hyperbrowser.ai/settings?tab=api_key). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY` .\n\n3\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/selenium\\#setup-browser-session) Setup Browser Session\n\nNext, you can easily startup a browser session using selenium and your Hyperbrowser API Key.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport dotenv from 'dotenv';\nimport https from 'https';\nimport { Builder, WebDriver } from 'selenium-webdriver';\nimport { Options } from 'selenium-webdriver/chrome';\nimport { Hyperbrowser } from '@hyperbrowser/sdk';\n\n// Load environment variables from .env file\ndotenv.config();\n\nconst client = new Hyperbrowser({ apiKey: process.env.HYPERBROWSER_API_KEY as string });\n\nasync function main() {\n const session = await client.sessions.create();\n\n const customHttpsAgent = new https.Agent({});\n (customHttpsAgent as any).addRequest = (req: any, options: any) => {\n req.setHeader('x-hyperbrowser-token', session.token);\n (https.Agent.prototype as any).addRequest.call(customHttpsAgent, req, options);\n };\n\n const driver: WebDriver = await new Builder()\n .forBrowser('chrome')\n .usingHttpAgent(customHttpsAgent)\n .usingServer('https://connect.hyperbrowser.ai/webdriver')\n .setChromeOptions(new Options())\n .build();\n\n try {\n // Navigate to a URL\n await driver.get(\"https://www.google.com\");\n console.log(\"Navigated to Google\");\n\n // Search\n const searchBox = await driver.findElement({ name: \"q\" });\n await searchBox.sendKeys(\"Selenium WebDriver\");\n await searchBox.submit();\n console.log(\"Performed search\");\n\n // Screenshot\n await driver.takeScreenshot().then(data => {\n require('fs').writeFileSync('search_results.png', data, 'base64');\n });\n console.log(\"Screenshot saved\");\n } finally {\n await driver.quit();\n }\n}\n\nif (require.main === module) {\n main().catch(console.error);\n}\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\n\nfrom selenium import webdriver\nfrom selenium.webdriver.remote.remote_connection import RemoteConnection\nfrom selenium.webdriver.chrome.options import Options\nfrom hyperbrowser import Hyperbrowser\n\n# Load environment variables from .env file\nload_dotenv()\n\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nclass CustomRC(RemoteConnection):\n _signing_key = None\n\n def __init__(self, server: str, token: str):\n super().__init__(server)\n self._token = token\n\n def get_remote_connection_headers(self, parsed_url, keep_alive=False):\n headers = super().get_remote_connection_headers(parsed_url, keep_alive)\n headers.update({'x-hyperbrowser-token': self._token})\n return headers\n\ndef main():\n session = client.sessions.create()\n custom_conn = CustomRC(\"https://connect.hyperbrowser.ai/webdriver\", session.token)\n driver = webdriver.Remote(custom_conn, options=Options())\n\n # Navigate to a URL\n driver.get(\"https://www.google.com\")\n print(\"Navigated to Google\")\n\n # Search\n search_box = driver.find_element(\"name\", \"q\")\n search_box.send_keys(\"Selenium WebDriver\")\n search_box.submit()\n print(\"Performed search\")\n\n # Screenshot\n driver.save_screenshot(\"search_results.png\")\n print(\"Screenshot saved\")\n\nif __name__ == \"__main__\":\n main()\n```\n\n4\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/get-started/quickstart/selenium\\#view-session-in-dashboard) View Session in Dashboard\n\nYou can view all your sessions in the [dashboard](https://app.hyperbrowser.ai/) and see their recordings or other key metrics like logs.\n\n[PreviousPlaywright](https://www.docs.hyperbrowser.ai/get-started/quickstart/playwright) [NextOverview](https://www.docs.hyperbrowser.ai/sessions/overview)\n\nLast updated 8 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/get-started/quickstart/selenium", "title": "Selenium | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/kb89O3th8srHD87rPAE9", "og:title": "Selenium | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/get-started/quickstart/selenium", "description": "Setup a browser session with Selenium.", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/kb89O3th8srHD87rPAE9", "twitter:title": "Selenium | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/get-started/quickstart/selenium", "og:description": "Setup a browser session with Selenium.", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Setup a browser session with Selenium." }, "html": null, "url": "https://www.docs.hyperbrowser.ai/get-started/quickstart/selenium", "status": "completed", "error": null }, "summary": "Selenium integration with Hyperbrowser allows for enhanced web scraping and automation capabilities." }, { "pathname": "/sessions/overview", "data": { "markdown": "Overview \\| Hyperbrowser\n\nIn Hyperbrowser, a Session is simply a dedicated, cloud-based browser instance that鈥檚 yours to direct. It鈥檚 like opening a fresh, private browser window鈥攐nly this one lives in the cloud and is fully controllable through code. Each Session keeps its own cookies, storage, and browsing context, so you can run your tasks without mixing them up.\n\nWith our Sessions API, you can spin up these browser environments whenever you need them, configure them with optional parameters, and close them down when you鈥檙e done. Higher level endpoints like scrape and crawl also utilize sessions internally to handle their tasks.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview\\#connect-with-your-favorite-browser-automation-libraries) Connect with your favorite browser automation libraries\n\n[**Puppeteer**\\\\\n\\\\\nConnect with Puppeteer to automate browser actions via a websocket connection](https://www.docs.hyperbrowser.ai/get-started/quickstart/puppeteer) [**Playwright**\\\\\n\\\\\nConnect with Playwright to automate browser actions via a websocket connection](https://www.docs.hyperbrowser.ai/get-started/quickstart/playwright)\n\nCheck out our [API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions) to see the Sessions API in more detail\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview\\#usage) Usage\n\nStarting a session is pretty simple, you can either startup a session directly by connecting to our websocket endpoint with your preferred automation tool like Playwright or Puppeteer, or you can create a session via the Sessions API.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview\\#simple-connect) Simple Connect\n\nWith this approach, you can startup a session with all default configurations set by just changing one line.\n\nNodePython\n\nPuppeteerPlaywright\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst browser = await puppeteer.connect({\n browserWSEndpoint: `wss://connect.hyperbrowser.ai?apiKey=${process.env.HYPERBROWSER_API_KEY}`,\n});\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst browser = await chromium.connectOverCDP(\n `wss://connect.hyperbrowser.ai?apiKey=${process.env.HYPERBROWSER_API_KEY}`\n);\n```\n\nPuppeteerPlaywright\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nbrowser = await pyppeteer.connect(\n browserWSEndpoint=f\"wss://connect.hyperbrowser.ai?apiKey={os.getenv('HYPERBROWSER_API_KEY')}\"\n)\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nwith sync_playwright() as p:\n browser = p.chromium.connect_over_cdp(\n f\"wss://connect.hyperbrowser.ai?apiKey={os.getenv('HYPERBROWSER_API_KEY')}\"\n )\n```\n\nThe simple connect approach doesn't support setting parameters. Use the Session API below for customizing session parameters, such as proxies, web recordings, or extensions.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview\\#connect-with-sessions-api) Connect with Sessions API\n\nWith this approach, you have more control over your sessions with additional features or updated configurations.\n\nNodePython\n\nPuppeteerPlaywright\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { connect } from \"puppeteer-core\";\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\"\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const session = await client.sessions.create({\n solveCaptchas: true,\n useStealth: true,\n useProxy: true,\n });\n const browser = await connect({\n browserWSEndpoint: session.wsEndpoint,\n });\n\n const [page] = await browser.pages();\n\n // Navigate to a website\n console.log(\"Navigating to Hacker News...\");\n await page.goto(\"https://news.ycombinator.com/\");\n const pageTitle = await page.title();\n console.log(\"Page 1:\", pageTitle);\n await page.evaluate(() => {\n console.log(\"Page 1:\", document.title);\n });\n\n // Clean up\n await page.close();\n await browser.close();\n await client.sessions.stop(session.id);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { chromium } from \"playwright-core\";\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const session = await client.sessions.create({\n solveCaptchas: true,\n useStealth: true,\n useProxy: true,\n });\n\n const browser = await chromium.connectOverCDP(session.wsEndpoint);\n\n const context = await browser.newContext();\n const page = await context.newPage();\n\n // Navigate to a website\n console.log(\"Navigating to Hacker News...\");\n await page.goto(\"https://news.ycombinator.com/\");\n const pageTitle = await page.title();\n console.log(\"Page 1:\", pageTitle);\n await page.evaluate(() => {\n console.log(\"Page 1:\", document.title);\n });\n\n // Clean up\n await page.close();\n await context.close();\n await browser.close();\n await client.sessions.stop(session.id);\n};\n\nmain();\n```\n\nPuppeteerPlaywright\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nfrom pyppeteer import connect\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser\nfrom hyperbrowser.models.session import CreateSessionParams\n\n# Load environment variables from .env file\nload_dotenv()\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n # Create a session and connect to it using Pyppeteer\n session = await client.sessions.create(\n params=CreateSessionParams(\n solve_captchas=True, use_stealth=True, use_proxy=True\n )\n )\n browser = await connect(browserWSEndpoint=session.ws_endpoint)\n\n pages = await browser.pages()\n page = pages[0]\n\n # Navigate to a website\n print(\"Navigating to Hacker News...\")\n await page.goto(\"https://news.ycombinator.com/\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n await page.evaluate(\"() => { console.log('Page 1:', document.title); }\")\n\n # Clean up\n await browser.close()\n await client.sessions.stop(session.id)\n\n# Run the async main function\nif __name__ == \"__main__\":\n asyncio.run(main())\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nfrom playwright.async_api import async_playwright\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser\n# You can also use the sync `Hyperbrowser` to work with sync_playwright\nfrom hyperbrowser.models.session import CreateSessionParams\n\n# Load environment variables from .env file\nload_dotenv()\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n # Create a session and connect to it using Playwright\n session = await client.sessions.create(\n params=CreateSessionParams(\n solve_captchas=True, use_stealth=True, use_proxy=True\n )\n )\n\n # Initialize Playwright and connect to the browser\n async with async_playwright() as p:\n browser = await p.chromium.connect_over_cdp(session.ws_endpoint)\n default_context = browser.contexts[0]\n page = default_context.pages[0]\n\n # Navigate to a website\n print(\"Navigating to Hacker News...\")\n await page.goto(\"https://news.ycombinator.com/\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n await page.evaluate(\"() => { console.log('Page 1:', document.title); }\")\n\n # Clean up\n await browser.close()\n await client.sessions.stop(session.id)\n\n# Run the async main function\nif __name__ == \"__main__\":\n asyncio.run(main())\n```\n\nHyperbrowser's CAPTCHA solving and Proxy features require being on a `PAID` plan.\n\n[PreviousSelenium](https://www.docs.hyperbrowser.ai/get-started/quickstart/selenium) [NextSession Parameters](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters)\n\nLast updated 8 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/sessions/overview", "title": "Overview | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/j9NqOR8L9oPVGxPJD2gI", "og:title": "Overview | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/sessions/overview", "description": "Learn about Hyperbrowser Sessions", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/j9NqOR8L9oPVGxPJD2gI", "twitter:title": "Overview | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/sessions/overview", "og:description": "Learn about Hyperbrowser Sessions", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Learn about Hyperbrowser Sessions" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/sessions/overview", "status": "completed", "error": null }, "summary": "Overview of Hyperbrowser, a tool for web scraping and automation, detailing its features and functionalities." }, { "pathname": "/sessions/overview/session-parameters", "data": { "markdown": "Session Parameters \\| Hyperbrowser\n\nSession Parameters are shared between the:\n\n- New Session\n\n- Scrape\n\n- Crawl\n\n- Extract\n\n- Agent Tasks\n\n\nThis is a comprehensive list and description of all parameters available when setting up a session.\n\nProxy Usage and CAPTCHA solving require being on a PAID plan.\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#enabling-stealth-mode-with-usestealth) **Enabling Stealth Mode with** `useStealth`\n\n- **Type**: `boolean`\n\n- **Description**: When set to `true`, the session will be launched in stealth mode, which employs various techniques to make the browser harder to detect as an automated tool.\n\n- **Default**: `false`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#using-a-proxy-with-useproxy) **Using a Proxy with** `useProxy`\n\n- **Type**: `boolean`\n\n- **Description**: When set to `true`, the session will be launched with a proxy server.\n\n- **Default**: `false`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#specifying-a-custom-proxy-server-with-proxyserver) **Specifying a Custom Proxy Server with** `proxyServer`\n\n- **Type**: `string`\n\n- **Description**: The hostname or IP address of the proxy server to use for the session. This option is only used when `useProxy` is set to `true`.\n\n- **Default**: `undefined`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#providing-proxy-server-authentication-with-proxyserverusername-and-proxyserverpassword) **Providing Proxy Server Authentication with** `proxyServerUsername` **and** `proxyServerPassword`\n\n- **Type**: `string`\n\n- **Description**: The username and password to use for authenticating with the proxy server, if required. These options are only used when `useProxy` is set to `true` and the proxy server requires authentication.\n\n- **Default**: `undefined`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#selecting-a-proxy-location-country-with-proxycountry) **Selecting a Proxy Location Country with** `proxyCountry`\n\n- **Type**: `string`\n\n- **Enum**: `[\"US\", \"GB\", \"CA\", ...]`\n\n- **Description**: The country where the proxy server should be located.\n\n- **Default**: `\"US\"`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#specifying-operating-systems-with-operatingsystems) **Specifying Operating Systems with** `operatingSystems`\n\n- **Type**: `array`\n\n- **Items**: `string`\n\n- **Enum**: `[\"windows\", \"android\", \"macos\", \"linux\", \"ios\"]`\n\n- **Description**: An array of operating systems to use for fingerprinting.\n\n- **Default**: `undefined`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#choosing-device-types-with-device) **Choosing Device Types with** `device`\n\n- **Type**: `array`\n\n- **Items**: `string`\n\n- **Enum**: `[\"desktop\", \"mobile\"]`\n\n- **Description**: An array of device types to use for fingerprinting.\n\n- **Default**: `undefined`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#selecting-browser-platforms-with-platform) **Selecting Browser Platforms with** `platform`\n\n- **Type**: `array`\n\n- **Items**: `string`\n\n- **Enum**: `[\"chrome\", \"firefox\", \"safari\", \"edge\"]`\n\n- **Description**: An array of browser platforms to use for fingerprinting.\n\n- **Default**: `undefined`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#setting-browser-locales-with-locales) **Setting Browser Locales with** `locales`\n\n- **Type**: `array`\n\n- **Items**: `string`\n\n- **Enum**: `[\"en\", \"es\", \"fr\", ...]`\n\n- **Description**: An array of browser locales to specify the language for the browser.\n\n- **Default**: `[\"en\"]`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#customizing-screen-resolution-with-screen) **Customizing Screen Resolution with** `screen`\n\n- **Type**: `object`\n\n- **Properties**:\n\n\n\n- `width` (number, default 1280): The screen width in pixels.\n\n- `height` (number, default 720): The screen height in pixels.\n\n\n- **Description**: An object specifying the screen resolution to emulate in the session.\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#solving-captchas-automatically-with-solvecaptchas) **Solving CAPTCHAs Automatically with** `solveCaptchas`\n\n- **Type**: `boolean`\n\n- **Description**: When set to `true`, the session will attempt to automatically solve any CAPTCHAs encountered during the session.\n\n- **Default**: `false`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#blocking-a-ds-with-adblock) **Blocking Ads with** `adblock`\n\n- **Type**: `boolean`\n\n- **Description**: When set to `true`, the session will attempt to block ads and other unwanted content during the session.\n\n- **Default**: `false`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#blocking-trackers-with-trackers) **Blocking Trackers with** `trackers`\n\n- **Type**: `boolean`\n\n- **Description**: When set to `true`, the session will attempt to block web trackers and other privacy-invasive technologies during the session.\n\n- **Default**: `false`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#blocking-annoyances-with-annoyances) **Blocking Annoyances with** `annoyances`\n\n- **Type**: `boolean`\n\n- **Description**: When set to `true`, the session will attempt to block common annoyances like pop-ups, overlays, and other disruptive elements during the session.\n\n- **Default**: `false`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#automatically-accepting-cookies-with-acceptcookies) Automatically accepting cookies with `acceptCookies`\n\n- **Type**: `boolean`\n\n- **Description**: When set to `true`, the session will attempt to accept all cookies on sites that are visited.\n\n- **Default**: `false`\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#enabling-web-recording-with-enablewebrecording) Enabling web recording with `enableWebRecording`\n\n- **Type**: `boolean`\n\n- **Description**: When set to `true`, the session will be recorded using [rrweb](https://www.rrweb.io/).\n\n- **Default**: `true`\n\n- [Learn more](https://www.docs.hyperbrowser.ai/sessions/recordings)\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters\\#reuse-browser-state-cookies-local-storage-etc.-across-sessions-with-profile) Reuse browser state(cookies, local storage, etc.) across sessions with `profile`\n\n- **Type**: `object`\n\n- **Properties**:\n\n\n\n- `id`\n\n\n\n- **Type**: `string`\n\n- **Description**: ID of the profile to use for the browser session.\n\n\n- `persistChanges`\n\n\n\n- **Type**: `boolean`\n\n- **Description**: When set to `true`, changes will be persisted to this profile on session close.\n\n\n- **Description**: An object specifying the profile configuration.\n\n- [Learn more](https://www.docs.hyperbrowser.ai/sessions/profiles)\n\n\n[PreviousOverview](https://www.docs.hyperbrowser.ai/sessions/overview) [NextAdvanced Privacy & Anti-Detection](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection)\n\nLast updated 7 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/sessions/overview/session-parameters", "title": "Session Parameters | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/LfGymF6rOOcBT91ZsRR1", "og:title": "Session Parameters | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/LfGymF6rOOcBT91ZsRR1", "twitter:title": "Session Parameters | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/sessions/overview/session-parameters", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters", "status": "completed", "error": null }, "summary": "This page details session parameters for configuring Hyperbrowser's web scraping and automation features." }, { "pathname": "/sessions/advanced-privacy-and-anti-detection", "data": { "markdown": "Advanced Privacy & Anti-Detection \\| Hyperbrowser\n\nSometimes you need your automated browser sessions to fly under the radar. With Hyperbrowser鈥檚 privacy and anti-detection features, you can tweak things like browser fingerprints, pick the right proxies, and decide exactly how requests get routed. This helps your automated visits look and feel more like regular browsing鈥攕omething that鈥檚 especially handy if you鈥檙e dealing with strict anti-bot measures or running sensitive operations.\n\nWhether you need to appear as if you鈥檙e browsing from a specific region, or you want to vary details like your device type and OS, it鈥檚 all possible. You can set things up so your workflows feel less like scripted tasks and more like genuine user behavior. Add in built-in captcha-solving capabilities, and you鈥檝e got a setup that keeps you moving forward, even if the sites you鈥檙e visiting throw a few hurdles your way.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection\\#stealth-mode) Stealth Mode\n\nStealth mode helps you avoid detection by anti-bot systems. It randomizes browser fingerprints and can be configured when creating a new session via the API. Options include:\n\n- **Devices** \\- Specify mobile or desktop device profiles\n\n- **Locales** \\- Set browser locale (e.g. en-US, fr-FR)\n\n- **Operating Systems** \\- Simulate different OSes like Android, iOS, Windows, macOS, Linux\n\n- **Screen Size** \\- Specify viewport dimensions to emulate different devices\n\n- **User Agents** \\- Rotate user agent strings\n\n\nTo enable stealth mode and other stealth configurations, you can set the desired options in the session creation params when creating a session.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { connect } from \"puppeteer-core\";\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const session = await client.sessions.create({\n useStealth: true,\n operatingSystems: [\"macos\"],\n device: [\"desktop\"],\n locales: [\"en\"],\n screen: {\n width: 1920,\n height: 1080,\n },\n });\n const browser = await connect({\n browserWSEndpoint: session.wsEndpoint,\n });\n\n const [page] = await browser.pages();\n\n // Navigate to a website\n console.log(\"Navigating to Hacker News...\");\n await page.goto(\"https://news.ycombinator.com/\");\n const pageTitle = await page.title();\n console.log(\"Page 1:\", pageTitle);\n await page.evaluate(() => {\n console.log(\"Page 1:\", document.title);\n });\n\n // Clean up\n await page.close();\n await browser.close();\n await client.sessions.stop(session.id);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nfrom pyppeteer import connect\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser\nfrom hyperbrowser.models.session import CreateSessionParams, ScreenConfig\n\n# Load environment variables from .env file\nload_dotenv()\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n # Create a session and connect to it using Pyppeteer\n session = await client.sessions.create(\n params=CreateSessionParams(\n use_stealth=True,\n operating_systems=[\"macos\"],\n device=[\"desktop\"],\n locales=[\"en\"],\n screen=ScreenConfig(width=1920, height=1080),\n )\n )\n browser = connect(browserWSEndpoint=session.ws_endpoint)\n\n pages = await browser.pages()\n page = pages[0]\n\n # Navigate to a website\n print(\"Navigating to Hacker News...\")\n await page.goto(\"https://news.ycombinator.com/\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n await page.evaluate(\"() => { console.log('Page 1:', document.title); }\")\n\n # Clean up\n await page.close()\n await browser.close()\n await client.sessions.stop(session.id)\n\n# Run the async main function\nif __name__ == \"__main__\":\n asyncio.get_event_loop().run_until_complete(main())\n```\n\nTo see all the available options, check out the [Create Session API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions#api-session)\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection\\#proxies) Proxies\n\nRoute browser traffic through proxies to change IP addresses. You can:\n\n- Use Hyperbrowser's proxy pool\n\n- Bring your own proxies\n\n\nTo enable these proxy configurations, you can set them in the session creation params.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { connect } from \"puppeteer-core\";\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const session = await client.sessions.create({\n useProxy: true,\n proxyCountry: \"US\",\n // use own proxy\n proxyServer: \"...\",\n proxyServerUsername: \"...\",\n proxyServerPassword: \"...\",\n });\n const browser = await connect({\n browserWSEndpoint: session.wsEndpoint,\n });\n\n const [page] = await browser.pages();\n\n // Navigate to a website\n console.log(\"Navigating to Hacker News...\");\n await page.goto(\"https://news.ycombinator.com/\");\n const pageTitle = await page.title();\n console.log(\"Page 1:\", pageTitle);\n await page.evaluate(() => {\n console.log(\"Page 1:\", document.title);\n });\n\n // Clean up\n await page.close();\n await browser.close();\n await client.sessions.stop(session.id);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nfrom pyppeteer import connect\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser\nfrom hyperbrowser.models.session import CreateSessionParams\n\n# Load environment variables from .env file\nload_dotenv()\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n # Create a session and connect to it using Pyppeteer\n session = await client.sessions.create(\n params=CreateSessionParams(\n use_proxy=True,\n proxy_country=\"US\",\n # use own proxy\n proxy_server=\"...\",\n proxy_server_username=\"...\",\n proxy_server_password=\"...\",\n )\n )\n browser = connect(browserWSEndpoint=session.ws_endpoint)\n\n pages = await browser.pages()\n page = pages[0]\n\n # Navigate to a website\n print(\"Navigating to Hacker News...\")\n await page.goto(\"https://news.ycombinator.com/\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n await page.evaluate(\"() => { console.log('Page 1:', document.title); }\")\n\n # Clean up\n await page.close()\n await browser.close()\n await client.sessions.stop(session.id)\n\n# Run the async main function\nif __name__ == \"__main__\":\n asyncio.get_event_loop().run_until_complete(main())\n```\n\nUsing proxies can introduce latency to any page navigations, so make sure to properly await these navigations with timeouts.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection\\#geospecific-proxies) Geospecific Proxies\n\nHyperbrowser supports geo-specific IPs as well. The specificity can range from country level down to city level. This can be specified by\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n...\nawait client.sessions.create({\n useProxy: true,\n proxyCountry: \"US\",\n // or proxyCity: \"New York\"\n });\n...\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n...\nclient.sessions.create(\n params=CreateSessionParams(\n use_proxy=True,\n proxy_country=\"US\",\n # or proxy_country=\"New York\"\n )\n)\n...\n```\n\nWhile Hyperbrowser supports nearly all countries and most cities, not all areas are guaranteed to be available. Especially for areas with low population density, separate testing should be done to ensure that adequate coverage is available.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection\\#custom-proxies) Custom Proxies\n\nIf you want to use your own proxy service/providers, you can do so by specifying the following parameters when creating a session. If you use your own proxy provider, you will not be billed for any proxy usage.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n...\nawait client.sessions.create({\n useProxy: true,\n proxyServer: \"subdomain.proxyprovider.io:10000\",\n proxyServerUsername: \"proxy-username\",\n proxyServerPassword: \"proxy-password\",\n });\n...\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n...\nclient.sessions.create(\n params=CreateSessionParams(\n use_proxy=True,\n proxy_server=\"subdomain.proxyprovider.io:10000\",\n proxy_server_username=\"proxy-username\",\n proxy_server_password=\"proxy-password\",\n )\n)\n...\n```\n\nProxy usage is only available on the `PAID` plans.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection\\#static-ips) Static IPs\n\nStatic IP usage is another powerful feature of Hyperbrowser, allowing users to maintain a consistent IP address across sessions. This feature can be essential for activities requiring a stable connection identity.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { connect } from \"puppeteer-core\";\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const session = await client.sessions.create({\n static_ip_id: ,\n });\n const browser = await connect({\n browserWSEndpoint: session.wsEndpoint,\n });\n\n const [page] = await browser.pages();\n\n // Navigation\n console.log(\"Using Static IP to navigate to Hacker News...\");\n await page.goto(\"https://news.ycombinator.com/\");\n const pageTitle = await page.title();\n console.log(\"Page title:\", pageTitle);\n\n // Clean up\n await page.close();\n await browser.close();\n await client.sessions.stop(session.id);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nfrom pyppeteer import connect\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser\nfrom hyperbrowser.models.session import CreateSessionParams\n\n# Load environment variables from .env file\nload_dotenv()\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n # Create a session with Static IP\n session = await client.sessions.create(\n params=CreateSessionParams(\n use_static_ip=True,\n )\n )\n browser = connect(browserWSEndpoint=session.ws_endpoint)\n\n pages = await browser.pages()\n page = pages[0]\n\n # Navigation\n print(\"Using Static IP to navigate to Hacker News...\")\n await page.goto(\"https://news.ycombinator.com/\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n\n # Clean up\n await page.close()\n await browser.close()\n await client.sessions.stop(session.id)\n\n# Run the async main function\nif __name__ == \"__main__\":\n asyncio.get_event_loop().run_until_complete(main())\n```\n\nManagement of static IP and corresponding user must be performed on the client side. Hyperbrowser maintains no inherent association between users and static IPs.\n\nAccess to static IPs is only available on `PAID` plans at request. If you, or your team, require static IPs let us know.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection\\#captcha-solving) CAPTCHA Solving\n\nHyperbrowser automatically solves CAPTCHAs when the `solveCaptchas` parameter is set to true for session creation.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { connect } from \"puppeteer-core\";\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const session = await client.sessions.create({\n solveCaptchas: true,\n });\n const browser = await connect({\n browserWSEndpoint: session.wsEndpoint,\n });\n\n const [page] = await browser.pages();\n\n // Navigate to a website\n console.log(\"Navigating to Hacker News...\");\n await page.goto(\"https://news.ycombinator.com/\");\n const pageTitle = await page.title();\n console.log(\"Page 1:\", pageTitle);\n await page.evaluate(() => {\n console.log(\"Page 1:\", document.title);\n });\n\n // Clean up\n await page.close();\n await browser.close();\n await client.sessions.stop(session.id);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nfrom pyppeteer import connect\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser\nfrom hyperbrowser.models.session import CreateSessionParams\n\n# Load environment variables from .env file\nload_dotenv()\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n # Create a session and connect to it using Pyppeteer\n session = await client.sessions.create(\n params=CreateSessionParams(\n solve_captchas=True,\n )\n )\n browser = connect(browserWSEndpoint=session.ws_endpoint)\n\n pages = await browser.pages()\n page = pages[0]\n\n # Navigate to a website\n print(\"Navigating to Hacker News...\")\n await page.goto(\"https://news.ycombinator.com/\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n await page.evaluate(\"() => { console.log('Page 1:', document.title); }\")\n\n # Clean up\n await page.close()\n await browser.close()\n await client.sessions.stop(session.id)\n\n# Run the async main function\nif __name__ == \"__main__\":\n asyncio.get_event_loop().run_until_complete(main())\n```\n\nCaptcha solving can take a bit of time, so make sure to implement proper waiting strategies when navigating to pages that might contain CAPTCHAs:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nawait page.goto(\"https://news.ycombinator.com/\",\n { waitUntil: \"networkidle0\" }\n);\n\n// OR\nconst sleep = (ms) => new Promise((res) => setTimeout(res, ms));\n\nawait page.waitForNavigation({ waitUntil: \"networkidle0\"});\nawait sleep(15000);\n```\n\nCAPTCHA solving is only available on `PAID` plans.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection\\#a-d-blocking) Ad Blocking\n\nHyperbrowser's browser instances can automatically block ads and trackers. This improves page load times and further reduces detection risk. In addition to ads, Hyperbrowser allows you to block trackers and other annoyances including cookie notices.\n\nTo enable ad blocking, set the proper configurations in the session create parameters.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { connect } from \"puppeteer-core\";\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const session = await client.sessions.create({\n adblock: true,\n });\n const browser = await connect({\n browserWSEndpoint: session.wsEndpoint,\n });\n\n const [page] = await browser.pages();\n\n // Navigate to a website\n console.log(\"Navigating to Hacker News...\");\n await page.goto(\"https://news.ycombinator.com/\");\n const pageTitle = await page.title();\n console.log(\"Page 1:\", pageTitle);\n await page.evaluate(() => {\n console.log(\"Page 1:\", document.title);\n });\n\n // Clean up\n await page.close();\n await browser.close();\n await client.sessions.stop(session.id);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nfrom pyppeteer import connect\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser\nfrom hyperbrowser.models.session import CreateSessionParams\n\n# Load environment variables from .env file\nload_dotenv()\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n # Create a session and connect to it using Pyppeteer\n session = await client.sessions.create(\n params=CreateSessionParams(\n adblock=True,\n trackers=True,\n annoyances=True,\n # You must have trackers set to true to enable blocking annoyances and\n # adblock set to true to enable blocking trackers.\n )\n )\n browser = connect(browserWSEndpoint=session.ws_endpoint)\n\n pages = await browser.pages()\n page = pages[0]\n\n # Navigate to a website\n print(\"Navigating to Hacker News...\")\n await page.goto(\"https://news.ycombinator.com/\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n await page.evaluate(\"() => { console.log('Page 1:', document.title); }\")\n\n # Clean up\n await page.close()\n await browser.close()\n await client.sessions.stop(session.id)\n\n# Run the async main function\nif __name__ == \"__main__\":\n asyncio.get_event_loop().run_until_complete(main())\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection\\#trackers-and-annoyances) Trackers and Annoyances\n\nHyperbrowser can block some trackers automatically if the options are selected when creating a session. In addition, Hyperbrowser can also filter out some popups and cookie prompts if the the options are selected.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n...\nawait client.sessions.create({\n adblock: true,\n trackers: true,\n annoyances: true,\n // You must have trackers set to true to enable blocking annoyances and\n // adblock set to true to enable blocking trackers.\n });\n...\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n...\nawait client.sessions.create(\n params=CreateSessionParams(\n adblock=True,\n trackers=True,\n annoyances=True,\n )\n )\n...\n```\n\nTo enable trackers blocking, adblock must be enabled.\n\nTo enable annoyance blocking, adblock and trackers must be enabled\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection\\#automatically-accept-cookies) Automatically Accept cookies\n\nSome site prompt users for cookies in a particularly intrusive way for scraping. If the `acceptCookies` param is set, then Hyperbrowser will automatically accept cookies on the browsers behalf.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nawait client.sessions.create({\n acceptCookies:true\n });\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nawait client.sessions.create({\n accept_cookies:True\n});\n```\n\n[PreviousSession Parameters](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters) [NextProfiles](https://www.docs.hyperbrowser.ai/sessions/profiles)\n\nLast updated 8 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection", "title": "Advanced Privacy & Anti-Detection | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/OkbhUZrC8aDeC2Qj9HVo", "og:title": "Advanced Privacy & Anti-Detection | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection", "description": "Hyperbrowser Anti-Bot Detection", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/OkbhUZrC8aDeC2Qj9HVo", "twitter:title": "Advanced Privacy & Anti-Detection | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection", "og:description": "Hyperbrowser Anti-Bot Detection", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Hyperbrowser Anti-Bot Detection" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection", "status": "completed", "error": null }, "summary": "This page discusses Hyperbrowser's advanced privacy features and anti-detection capabilities for web scraping and automation." }, { "pathname": "/sessions/profiles", "data": { "markdown": "Profiles \\| Hyperbrowser\n\nHyperbrowser profiles allow you to reuse browser state like cookies, local storage, and network cache across multiple sessions. This can improve performance and enable seamless workflows requiring persistent data.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/profiles\\#how-profiles-work) How Profiles Work\n\nA profile is essentially a saved snapshot of a browser's user data directory. By default, each Hyperbrowser session uses a fresh user data directory to ensure isolation.\n\nWhen you create a profile and then use and persist it in a new session, Hyperbrowser saves that session's user data directory. You can then attach the profile to future sessions to pick up where you left off.\n\nCommon use cases for profiles include:\n\n- Reusing authenticated sessions\n\n- Improving page load times with a primed cache\n\n- Preserving application state across sessions\n\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/profiles\\#using-profiles) Using Profiles\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/profiles\\#id-1.-create-a-profile) 1\\. Create a Profile\n\nFirst, create a new profile using the Profiles API. Note the returned `id` \\- you'll need this to attach the profile to sessions.\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst profile = await client.profiles.create();\nconsole.log(\"profile\", profile.id);\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nprofile = client.profiles.create()\nprint(\"profile\", profile.id)\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST 'https://app.hyperbrowser.ai/api/profile' \\\n-H 'Content-Type: application/json' \\\n-H 'x-api-key: YOUR_API_KEY' \\\n-d '{}'\n```\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/profiles\\#id-2.-attach-profile-to-session) 2\\. Attach Profile to Session\n\nWhen creating a new session with the Sessions API, include the `id` in the `profile` field:\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst session = await client.sessions.create({\n profile: {\n id: \"\",\n persistChanges: true, // set to true for the first session of a new profile\n },\n});\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nsession = client.sessions.create(\n params=CreateSessionParams(\n profile=CreateSessionProfile(\n id=\"\",\n # set to true for the first session of a new profile\n persist_changes=True,\n ),\n )\n)\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST 'https://app.hyperbrowser.ai/api/session' \\\n-H 'Content-Type: application/json' \\\n-H 'x-api-key: YOUR_API_KEY' \\\n-d '{\n \"profile\": {\n \"id\": \"\"\n }\n}'\n```\n\nSet `\"persistChanges\": true` in the `profile` object if you want the session to update the profile with any changes to cookies, cache, etc. You will need to do this for the first time you use a new profile in a session so it can be used in subsequent sessions.\n\nOnce whatever changes to the profile have been made, the session should be safely closed to ensure that the profile is saved. After that unless the `\"persistChanges\": true` option is passed to the session again, the session will be immutable.\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/profiles\\#id-3.-reuse-profile) 3\\. Reuse Profile\n\nAttach the same `id` to additional sessions to reuse browser state. Created profiles are reusable until you delete them.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/profiles\\#deleting-profiles) Deleting Profiles\n\nDelete unused profiles to free up resources. Once deleted, a profile is no longer attachable to sessions.\n\nTo delete a profile, send a DELETE request to the Profiles API with the target `id`:\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.profiles.delete(\"\");\nconsole.log(\"response\", response);\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nresponse = client.profiles.delete(\"\")\nprint(\"response\", response)\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X DELETE 'https://app.hyperbrowser.ai/api/profile/' \\\n-H 'x-api-key: YOUR_API_KEY'\n```\n\nDeletion is irreversible, so be sure you no longer need a profile before deleting.\n\n[PreviousAdvanced Privacy & Anti-Detection](https://www.docs.hyperbrowser.ai/sessions/advanced-privacy-and-anti-detection) [NextRecordings](https://www.docs.hyperbrowser.ai/sessions/recordings)\n\nLast updated 7 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/sessions/profiles", "title": "Profiles | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/KT16iMv7GSEut0whWUzA", "og:title": "Profiles | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/sessions/profiles", "description": "Hyperbrowser Browser Profiles", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/KT16iMv7GSEut0whWUzA", "twitter:title": "Profiles | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/sessions/profiles", "og:description": "Hyperbrowser Browser Profiles", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Hyperbrowser Browser Profiles" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/sessions/profiles", "status": "completed", "error": null }, "summary": "The \"Profiles\" page in Hyperbrowser outlines how to manage user profiles for data scraping and automation tasks." }, { "pathname": "/sessions/recordings", "data": { "markdown": "Recordings \\| Hyperbrowser\n\nHyperbrowser allows you to record and replay your browser sessions. It uses [rrweb](https://www.rrweb.io/), an open-source web session replay library. Session recordings let you:\n\n- Visually debug test failures and errors\n\n- Analyze user behavior and interactions\n\n- Share reproducible bug reports\n\n- Save and archive session data\n\n\nYes, you can embed a looped video using HTML within your Markdown file. Here鈥檚 an example of how to do it:\n\n![](https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2Fhyperbrowser-assets-bucket.s3.us-east-1.amazonaws.com%2Fweb-recording-2.gif&width=768&dpr=4&quality=100&sign=ea03a722&sv=2)\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/recordings\\#enabling-session-recording) Enabling Session Recording\n\nTo record a session, set the `enableWebRecording` option to `true` when creating a new Hyperbrowser session:\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const session = await client.sessions.create({\n enableWebRecording: true,\n });\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser\nfrom hyperbrowser.models.session import CreateSessionParams\n\nload_dotenv()\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n session = await client.sessions.create(\n params=CreateSessionParams(\n enable_web_recording=True,\n )\n )\n\nif __name__ == \"__main__\":\n import asyncio\n\n asyncio.run(main())\n```\n\nThis will record all browser interactions, DOM changes, and network requests for the duration of the session.\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/recordings\\#retrieving-recordings) Retrieving Recordings\n\nPlease contact us at info@hyperbrowser.ai to get access to this feature.\n\n1. Note the `id` of the session you want to replay\n\n2. Use the Session Recordings API to download the recording, or if you are using the SDKs, you can just call the `getRecording` function:\n\n\nNodePythoncurl\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const recordingData = await client.sessions.getRecording(\n \"91e96d43-0dd2-4882-8d3a-613b12583ba2\"\n );\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import AsyncHyperbrowser\n\nload_dotenv()\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n recording_data = await client.sessions.get_recording(\n \"91e96d43-0dd2-4882-8d3a-613b12583ba2\"\n )\n\nif __name__ == \"__main__\":\n import asyncio\n\n asyncio.run(main())\n\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X GET 'https://app.hyperbrowser.ai/api/session/{sessionId}/recording' \\\n-H 'x-api-key: YOUR_API_KEY\n```\n\nThe recording data will be returned in [rrweb's JSON format](https://github.com/rrweb-io/rrweb/blob/98e71cd0d23628cd1fbdbe47664a65748084c4a4/packages/types/src/index.ts#L169).\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/recordings\\#replaying-recordings) Replaying Recordings\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/recordings\\#using-rrwebs-player) Using rrweb's Player\n\nHere's an example of using rrweb's player to replay a recording:\n\n1. Include the rrweb player script on your page:\n\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n\n\n```\n\n1. Add a container element for your player\n\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n
\n```\n\n1. Initialize the player with your recording data\n\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n// if using rrweb npm package\nimport rrwebPlayer from \"rrweb-player\";\nimport \"rrweb-player/dist/style.css\";\n\nconst recordingData = YOUR_RECORDING_DATA\n\nconst replayer = new rrwebPlayer({\n target: document.getElementById('player'),\n props: {\n events: recordingData,\n showController: true,\n autoPlay: true,\n },\n});\n```\n\nThis will launch an interactive player UI that allows you to play, pause, rewind, and inspect the recorded session.\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/recordings\\#building-a-custom-player) Building a custom player\n\nYou can also use rrweb's APIs to build your own playback UI. Refer to the [rrweb documentation](https://github.com/rrweb-io/rrweb/blob/master/guide.md#replay) for thorough details on how to customize the Replayer to your needs.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/recordings\\#storage-and-retention) Storage and Retention\n\nSession recordings are stored securely in Hyperbrowser's cloud infrastructure. Recordings are retained according to your plan's data retention policy.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/recordings\\#limitations) Limitations\n\n- Session recordings capture only the visual state of the page. They do not include server-side logs, database changes, or other non-DOM modifications.\n\n- Recordings may not perfectly reproduce complex WebGL or canvas-based animations.\n\n\n[PreviousProfiles](https://www.docs.hyperbrowser.ai/sessions/profiles) [NextLive View](https://www.docs.hyperbrowser.ai/sessions/live-view)\n\nLast updated 7 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/sessions/recordings", "title": "Recordings | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/uBLIvMMZffXKFpakh7Uj", "og:title": "Recordings | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/sessions/recordings", "description": "Hyperbrowser Session Recordings", "color-scheme": "dark", "link:preload": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2Fhyperbrowser-assets-bucket.s3.us-east-1.amazonaws.com%2Fweb-recording-2.gif&width=768&dpr=4&quality=100&sign=ea03a722&sv=2", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/uBLIvMMZffXKFpakh7Uj", "twitter:title": "Recordings | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/sessions/recordings", "og:description": "Hyperbrowser Session Recordings", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Hyperbrowser Session Recordings" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/sessions/recordings", "status": "completed", "error": null }, "summary": "The page covers Hyperbrowser's recording feature for efficient web scraping and automation processes." }, { "pathname": "/sessions/live-view", "data": { "markdown": "Live View \\| Hyperbrowser\n\nHyperbrowser's Live View feature allows you to observe and interact with your browser sessions in real-time. You can use Live View to:\n\n- Debug and troubleshoot your scripts\n\n- Monitor the progress of long-running automations\n\n- Provide a way for end-users to interact with the browser\n\n- Perform hybrid workflows, such as ones where human-in-loop interactions are required.\n\n\nYour browser does not support HTML5 videos.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/live-view\\#how-it-works) How it Works\n\nLive View works by providing a secure, authenticated URL that embeds a live stream of the browser session. The URL can be accessed in any modern web browser.\n\nWhenever you create a new session or get the details of an existing session, Hyperbrowser returns a `liveUrl` field with a unique URL tied to that specific session. The URL remains valid as long as the session is active and the token in the URL hasn't expired ( [Securing Live View](https://www.docs.hyperbrowser.ai/sessions/live-view#securing-live-view)).\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst session = await hbClient.sessions.create();\nconst liveUrl = session.liveUrl;\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nsession = hb_client.sessions.create()\nlive_url = session.live_url\n```\n\nThe returned `liveUrl` will look something like:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nhttps://app.hyperbrowser.ai/live?token=&keepAlive=true\n```\n\nYou can now open this URL in a web browser to view the live session.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/live-view\\#embedding-live-view) Embedding Live View\n\nYou can embed a Live View into your own web application using an iframe:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n\n```\n\nThis is useful for providing a seamless experience to your end-users. For example, you could use an embedded Live View to:\n\n- Show the progress of a web scraping job\n\n- Allow users to complete a complex workflow that requires manual intervention\n\n- Provide a preview of an automated process before committing it\n\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/live-view\\#securing-live-view) Securing Live View\n\nLive View URLs are secured with authentication and encryption. Only users with the correct URL can access the Live View.\n\nHowever, anyone with the URL can view (and potentially interact with) the session. Be sure to protect Live View URLs as sensitive secrets, especially if you're embedding them in a public web page.\n\nThe token in the `liveUrl` will expire after 12 hours. In this case, you can simply call the GET request for the Sessions API to get the details for the given session id which will also return a new `liveUrl` with a refreshed token.\n\n[PreviousRecordings](https://www.docs.hyperbrowser.ai/sessions/recordings) [NextExtensions](https://www.docs.hyperbrowser.ai/sessions/extensions)\n\nLast updated 7 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/sessions/live-view", "title": "Live View | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/YDJVsBykISUuW2ivppGv", "og:title": "Live View | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/sessions/live-view", "description": "Hyperbrowser Live View", "color-scheme": "dark", "link:preload": "https://cdn.iframe.ly/embed.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/YDJVsBykISUuW2ivppGv", "twitter:title": "Live View | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/sessions/live-view", "og:description": "Hyperbrowser Live View", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Hyperbrowser Live View" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/sessions/live-view", "status": "completed", "error": null }, "summary": "The Live View feature in Hyperbrowser allows real-time monitoring and interaction with web scraping tasks." }, { "pathname": "/sessions/extensions", "data": { "markdown": "Extensions \\| Hyperbrowser\n\nHyperbrowser allows you to enhance your browser sessions with custom Chrome extensions.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/extensions\\#uploading-an-extension) Uploading an Extension\n\nTo use a custom extension with Hyperbrowser:\n\n1. Create your extension as a directory containing at least a `manifest.json` file\n\n2. Compress the directory into a ZIP archive, make sure to zip all the files and not the directory itself\n\n3. Upload the ZIP file using the Extensions API:\n\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\nconfig();\n\nconst hbClient = new Hyperbrowser({ apiKey: process.env.HYPERBROWSER_API_KEY });\n\nconst uploadExtension = async () => {\n const resp = await hbClient.extensions.create({\n name: \"new-extension\",\n filePath: \"custom-extension.zip\",\n });\n console.log(resp);\n};\n\nuploadExtension();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nimport os\nfrom hyperbrowser import AsyncHyperbrowser\nfrom hyperbrowser.models.extension import CreateExtensionParams\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nhb_client = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def upload_extension():\n res = await hb_client.extensions.create(\n CreateExtensionParams(name=\"new-extension\", file_path=\"custom-extension.zip\")\n )\n print(res)\n\nif __name__ == \"__main__\":\n asyncio.run(upload_extension())\n```\n\nThe API will return an `id` that you'll use to attach the extension to a session.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/sessions/extensions\\#using-an-extension) Using an Extension\n\nTo use your uploaded extension in a Hyperbrowser session, simply add it in the `extensionIds` field of the Create Session params:\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\nconfig();\n\nconst hbClient = new Hyperbrowser({ apiKey: process.env.HYPERBROWSER_API_KEY });\n\nconst useExtensionInSession = async () => {\n const resp = await hbClient.sessions.create({\n extensionIds: [\"\"],\n });\n console.log(resp);\n};\n\nuseExtensionInSession();\n\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport asyncio\nimport os\nfrom hyperbrowser import AsyncHyperbrowser\nfrom hyperbrowser.models.session import CreateSessionParams\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nhb_client = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def use_extension_in_session():\n res = await hb_client.sessions.create(\n CreateSessionParams(extension_ids=[\"\"])\n )\n print(res)\n\nif __name__ == \"__main__\":\n asyncio.run(use_extensionin_session())\n\n```\n\nOnce added to a session, these extension behave exactly as they would in a real browser.\n\nRefer to the [Extensions API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/extensions) for more details on uploading and managing extensions.\n\n[PreviousLive View](https://www.docs.hyperbrowser.ai/sessions/live-view) [NextScrape](https://www.docs.hyperbrowser.ai/web-scraping/scrape)\n\nLast updated 8 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/sessions/extensions", "title": "Extensions | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/4JcXxc69pQLgL80Kxno0", "og:title": "Extensions | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/sessions/extensions", "description": "Using Extensions with Hyperbrowser", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/4JcXxc69pQLgL80Kxno0", "twitter:title": "Extensions | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/sessions/extensions", "og:description": "Using Extensions with Hyperbrowser", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Using Extensions with Hyperbrowser" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/sessions/extensions", "status": "completed", "error": null }, "summary": "The page discusses extensions for Hyperbrowser, enhancing its web scraping and automation capabilities." }, { "pathname": "/web-scraping/scrape", "data": { "markdown": "Scrape \\| Hyperbrowser\n\nThe Scrape API allows you to get the data you want from web pages using with a single call. You can scrape page content and capture it's data in various formats.\n\nFor detailed usage, checkout the [Scrape API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/scrape#api-scrape)\n\nHyperbrowser exposes endpoints for starting a scrape request and for getting it's status and results. By default, scraping is handled in an asynchronous manner of first starting the job and then checking it's status until it is completed. However, with our SDKs, we provide a simple function that handles the whole flow and returns the data once the job is completed.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/scrape\\#installation) Installation\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install hyperbrowser\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/scrape\\#usage) Usage\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n // Handles both starting and waiting for scrape job response\n const scrapeResult = await client.scrape.startAndWait({\n url: \"https://example.com\",\n });\n console.log(\"Scrape result:\", scrapeResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.scrape import StartScrapeJobParams\n\n# Load environment variables from .env file\nload_dotenv()\n\n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\n# Start scraping and wait for completion\nscrape_result = client.scrape.start_and_wait(\n StartScrapeJobParams(url=\"https://example.com\")\n)\nprint(\"Scrape result:\", scrape_result)\n```\n\nStart Scrape Job\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/scrape \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"url\": \"https://example.com\"\n }'\n```\n\nGet Scrape Job Status and Data\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl https://app.hyperbrowser.ai/api/scrape/{jobId} \\\n -H 'x-api-key: '\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/scrape\\#response) Response\n\nThe Start Scrape Job `POST /scrape` endpoint will return a `jobId` in the response which can be used to get information about the job in subsequent requests.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"962372c4-a140-400b-8c26-4ffe21d9fb9c\"\n}\n```\n\nThe Get Scrape Job `GET /scrape/{jobId}` will return the following data:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"962372c4-a140-400b-8c26-4ffe21d9fb9c\",\n \"status\": \"completed\",\n \"data\": {\n \"metadata\": {\n \"title\": \"Example Page\",\n \"description\": \"A sample webpage\"\n },\n \"markdown\": \"# Example Page\\nThis is content...\",\n }\n}\n```\n\nThe status of a scrape job can be one of `pending`, `running`, `completed`, `failed` . There can also be other optional fields like `error` with an error message if an error was encountered, and `html` and `links` in the data object depending on which formats are requested for the request.\n\nTo see the full schema, checkout the [API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/scrape#api-scrape-id).\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/scrape\\#session-configurations) Session Configurations\n\nYou can also provide configurations for the session that will be used to execute the scrape job just as you would when creating a new session itself. These could include using a proxy or solving CAPTCHAs.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const scrapeResult = await client.scrape.startAndWait({\n url: \"https://example.com\",\n sessionOptions: {\n useProxy: true,\n solveCaptchas: true,\n proxyCountry: \"US\",\n locales: [\"en\"],\n },\n });\n console.log(\"Scrape result:\", scrapeResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.scrape import StartScrapeJobParams\nfrom hyperbrowser.models.session import CreateSessionParams\n\n# Load environment variables from .env file\nload_dotenv()\n\n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\n# Start scraping and wait for completion\nscrape_result = client.scrape.start_and_wait(\n StartScrapeJobParams(\n url=\"https://example.com\",\n session_options=CreateSessionParams(use_proxy=True, solve_captchas=True),\n )\n)\nprint(\"Scrape result:\", scrape_result)\n```\n\nUsing proxy and solving CAPTCHAs will slow down the scrape so use it if necessary.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/scrape\\#scrape-configurations) Scrape Configurations\n\nYou can also provide optional parameters for the scrape job itself such as the formats to return, only returning the main content of the page, setting the maximum timeout for navigating to a page, etc.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const scrapeResult = await client.scrape.startAndWait({\n url: \"https://example.com\",\n scrapeOptions: {\n formats: [\"markdown\", \"html\", \"links\"],\n onlyMainContent: false,\n timeout: 15000,\n },\n });\n console.log(\"Scrape result:\", scrapeResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.scrape import ScrapeOptions, StartScrapeJobParams\n\n# Load environment variables from .env file\nload_dotenv()\n\n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\n# Start scraping and wait for completion\nscrape_result = client.scrape.start_and_wait(\n StartScrapeJobParams(\n url=\"https://example.com\",\n scrape_options=ScrapeOptions(\n formats=[\"html\", \"links\", \"markdown\"], only_main_content=False, timeout=5000\n ),\n )\n)\nprint(\"Scrape result:\", scrape_result)\n```\n\nFor a full reference on the scrape endpoint, checkout the [API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/scrape), or read the [Advanced Scraping Guide](https://www.docs.hyperbrowser.ai/guides/scraping) to see more advanced options for scraping.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/scrape\\#batch-scrape) Batch Scrape\n\nBatch Scrape works the same as regular scrape, except instead of a single URL, you can provide a list of up to 1,000 URLs to scrape at once.\n\nBatch Scrape is currently only available on the `Ultra` plan.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const scrapeResult = await client.scrape.batch.startAndWait({\n urls: [\"https://example.com\", \"https://hyperbrowser.ai\"],\n scrapeOptions: {\n formats: [\"markdown\", \"html\", \"links\"],\n },\n });\n console.log(\"Scrape result:\", scrapeResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.scrape import ScrapeOptions, StartBatchScrapeJobParams\n\nload_dotenv()\n\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nscrape_result = client.scrape.batch.start_and_wait(\n StartBatchScrapeJobParams(\n urls=[\"https://example.com\", \"https://hyperbrowser.ai\"],\n scrape_options=ScrapeOptions(\n formats=[\"html\", \"links\", \"markdown\"]\n ),\n )\n)\nprint(\"Scrape result:\", scrape_result)\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/scrape\\#response-1) Response\n\nThe Start Batch Scrape Job `POST /scrape/batch` endpoint will return a `jobId` in the response which can be used to get information about the job in subsequent requests.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"962372c4-a140-400b-8c26-4ffe21d9fb9c\"\n}\n```\n\nThe Get Batch Scrape Job `GET /scrape/batch/{jobId}` will return the following data:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"962372c4-a140-400b-8c26-4ffe21d9fb9c\",\n \"status\": \"completed\",\n \"totalScrapedPages\": 2,\n \"totalPageBatches\": 1,\n \"currentPageBatch\": 1,\n \"batchSize\": 20,\n \"data\": [\\\n {\\\n \"markdown\": \"Hyperbrowser\\n\\n[Home](https://hyperbrowser.ai/)...\",\\\n \"metadata\": {\\\n \"url\": \"https://www.hyperbrowser.ai/\",\\\n \"title\": \"Hyperbrowser\",\\\n \"viewport\": \"width=device-width, initial-scale=1\",\\\n \"link:icon\": \"https://www.hyperbrowser.ai/favicon.ico\",\\\n \"sourceURL\": \"https://hyperbrowser.ai\",\\\n \"description\": \"Infinite Browsers\"\\\n },\\\n \"url\": \"hyperbrowser.ai\",\\\n \"status\": \"completed\",\\\n \"error\": null\\\n },\\\n {\\\n \"markdown\": \"Example Domain\\n\\n# Example Domain...\",\\\n \"metadata\": {\\\n \"url\": \"https://www.example.com/\",\\\n \"title\": \"Example Domain\",\\\n \"viewport\": \"width=device-width, initial-scale=1\",\\\n \"sourceURL\": \"https://example.com\"\\\n },\\\n \"url\": \"example.com\",\\\n \"status\": \"completed\",\\\n \"error\": null\\\n }\\\n ]\n}\n```\n\nThe status of a batch scrape job can be one of `pending`, `running`, `completed`, `failed` . The results of all the scrapes will be an array in the `data` field of the response. Each scraped page will be returned in the order of the initial provided urls, and each one will have its own status and information.\n\nTo see the full schema, checkout the [API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/scrape#api-scrape-batch-id).\n\nAs with the single scrape, by default, batch scraping is handled in an asynchronous manner of first starting the job and then checking it's status until it is completed. However, with our SDKs, we provide a simple function ( `client.scrape.batch.startAndWait`) that handles the whole flow and returns the data once the job is completed.\n\n[PreviousExtensions](https://www.docs.hyperbrowser.ai/sessions/extensions) [NextCrawl](https://www.docs.hyperbrowser.ai/web-scraping/crawl)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/web-scraping/scrape", "title": "Scrape | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/kHqG5hklpBJnkwD7io43", "og:title": "Scrape | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/web-scraping/scrape", "description": "Scrape any page and get formatted data", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/kHqG5hklpBJnkwD7io43", "twitter:title": "Scrape | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/web-scraping/scrape", "og:description": "Scrape any page and get formatted data", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Scrape any page and get formatted data" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/web-scraping/scrape", "status": "completed", "error": null }, "summary": "\"Scrape\" page in Hyperbrowser documentation focuses on scraping data from web pages using the tool's features." }, { "pathname": "/web-scraping/crawl", "data": { "markdown": "Crawl \\| Hyperbrowser\n\nThe Crawl API allows you to crawl through an entire website and get all it's data with a single call.\n\nFor detailed usage, checkout the [Crawl API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/crawl)\n\nHyperbrowser exposes endpoints for starting a crawl request and for getting it's status and results. By default, crawling is handled in an asynchronous manner of first starting the job and then checking it's status until it is completed. However, with our SDKs, we provide a simple function that handles the whole flow and returns the data once the job is completed.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/crawl\\#installation) Installation\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install hyperbrowser\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/crawl\\#usage) Usage\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n // Handles both starting and waiting for crawl job response\n const crawlResult = await client.crawl.startAndWait({\n url: \"https://hyperbrowser.ai\",\n });\n console.log(\"Crawl result:\", crawlResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.crawl import StartCrawlJobParams\n\n# Load environment variables from .env file\nload_dotenv()\n\n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\n# Start crawling and wait for completion\ncrawl_result = client.crawl.start_and_wait(\n StartCrawlJobParams(url=\"https://hyperbrowser.ai\")\n)\nprint(\"Crawl result:\", crawl_result)\n```\n\nStart Crawl Job\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/crawl \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"url\": \"https://hyperbrowser.ai\"\n }'\n```\n\nGet Crawl Job Status and Data\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl https://app.hyperbrowser.ai/api/crawl/{jobId} \\\n -H 'x-api-key: '\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/crawl\\#response) Response\n\nThe Start Crawl Job `POST /crawl` endpoint will return a `jobId` in the response which can be used to get information about the job in subsequent requests.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"962372c4-a140-400b-8c26-4ffe21d9fb9c\"\n}\n```\n\nThe Get Crawl Job `GET /crawl/{jobId}` will return the following data:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"962372c4-a140-400b-8c26-4ffe21d9fb9c\",\n \"status\": \"completed\",\n \"totalCrawledPages\": 2,\n \"totalPageBatches\": 1,\n \"currentPageBatch\": 1,\n \"batchSize\": 20,\n \"data\": [\\\n {\\\n \"url\": \"https://example.com\",\\\n \"status\": \"completed\",\\\n \"metadata\": {\\\n \"title\": \"Example Page\",\\\n \"description\": \"A sample webpage\"\\\n },\\\n \"markdown\": \"# Example Page\\nThis is content...\",\\\n },\\\n ...\\\n ]\n}\n```\n\nThe status of a crawl job can be one of `pending`, `running`, `completed`, `failed` . There can also be other optional fields like `error` with an error message if an error was encountered, and `html` and `links` in the data object depending on which formats are requested for the request.\n\nUnlike the scrape endpoint, the crawl endpoint returns a list in the data field with the all the pages that were crawled in the current page batch. The SDKs also provide a function which will start the crawl job, wait until it's complete, and return all the crawled pages for the entire crawl.\n\nEach crawled page has it's own status of `completed` or `failed` and can have it's own error field, so be cautious of that.\n\nTo see the full schema, checkout the [API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/crawl#api-crawl-id).\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/crawl\\#additional-crawl-configurations) Additional Crawl Configurations\n\nThe crawl endpoint provides additional parameters you can provide to tailor the crawl to your needs. You can narrow down the pages crawled by setting a limit to the maximum number of pages visited, only including paths that match a certain pattern, excluding paths that match another pattern, etc.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n // Handles both starting and waiting for crawl job response\n const crawlResult = await client.crawl.startAndWait({\n url: \"https://hyperbrowser.ai\",\n maxPages: 5,\n includePatterns: [\"/blogs/*\"],\n });\n console.log(\"Crawl result:\", crawlResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.crawl import StartCrawlJobParams\n\n# Load environment variables from .env file\nload_dotenv()\n\n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\n# Start crawling and wait for completion\ncrawl_result = client.crawl.start_and_wait(\n StartCrawlJobParams(\n url=\"https://hyperbrowser.ai\",\n max_pages=5,\n include_patterns: [\"/blogs/*\"],\n )\n)\nprint(\"Crawl result:\", crawl_result)\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/crawl\\#session-configurations) Session Configurations\n\nYou can also provide configurations for the session that will be used to execute the crawl job just as you would when creating a new session itself. These could include using a proxy or solving CAPTCHAs.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const crawlResult = await client.crawl.startAndWait({\n url: \"https://example.com\",\n sessionOptions: {\n useProxy: true,\n solveCaptchas: true,\n proxyCountry: \"US\",\n locales: [\"en\"],\n },\n });\n console.log(\"Crawl result:\", crawlResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.crawl import StartCrawlJobParams\nfrom hyperbrowser.models.session import CreateSessionParams\n\n# Load environment variables from .env file\nload_dotenv()\n\n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\n# Start crawling and wait for completion\ncrawl_result = client.crawl.start_and_wait(\n StartCrawlJobParams(\n url=\"https://example.com\",\n session_options=CreateSessionParams(use_proxy=True, solve_captchas=True),\n )\n)\nprint(\"Crawl result:\", crawl_result)\n```\n\nUsing proxy and solving CAPTCHAs will slow down the crawl so use it only if necessary.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/crawl\\#scrape-configurations) Scrape Configurations\n\nYou can also provide optional scrape options for the crawl job such as the formats to return, only returning the main content of the page, setting the maximum timeout for navigating to a page, etc.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const crawlResult = await client.crawl.startAndWait({\n url: \"https://example.com\",\n scrapeOptions: {\n formats: [\"markdown\", \"html\", \"links\"],\n onlyMainContent: false,\n timeout: 10000,\n },\n });\n console.log(\"Crawl result:\", crawlResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.scrape import ScrapeOptions\nfrom hyperbrowser.models.crawl import StartCrawlJobParams\n\n# Load environment variables from .env file\nload_dotenv()\n\n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\n# Start crawling and wait for completion\ncrawl_result = client.crawl.start_and_wait(\n StartCrawlJobParams(\n url=\"https://example.com\",\n scrape_options=ScrapeOptions(\n formats=[\"html\", \"links\", \"markdown\"], only_main_content=False, timeout=10000\n ),\n )\n)\nprint(\"Crawl result:\", crawl_result)\n```\n\nFor a full reference on the crawl endpoint, checkout the [API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/crawl), or read the [Advanced Scraping Guide](https://www.docs.hyperbrowser.ai/guides/scraping) to see more advanced options for scraping.\n\n[PreviousScrape](https://www.docs.hyperbrowser.ai/web-scraping/scrape) [NextExtract](https://www.docs.hyperbrowser.ai/web-scraping/extract)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/web-scraping/crawl", "title": "Crawl | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/CV0PDNdR79Ma43fRtcuC", "og:title": "Crawl | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/web-scraping/crawl", "description": "Crawl a website and it's links to extract all it's data", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/CV0PDNdR79Ma43fRtcuC", "twitter:title": "Crawl | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/web-scraping/crawl", "og:description": "Crawl a website and it's links to extract all it's data", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Crawl a website and it's links to extract all it's data" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/web-scraping/crawl", "status": "completed", "error": null }, "summary": "The page discusses how to utilize Hyperbrowser for effective web crawling." }, { "pathname": "/web-scraping/extract", "data": { "markdown": "Extract \\| Hyperbrowser\n\nThe Extract API allows you to get data in a structured format for any provided URLs with a single call.\n\nFor detailed usage, checkout the [Extract API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/extract)\n\nHyperbrowser exposes endpoints for starting an extract request and for getting it's status and results. By default, extracting is handled in an asynchronous manner of first starting the job and then checking it's status until it is completed. However, with our SDKs, we provide a simple function that handles the whole flow and returns the data once the job is completed.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/extract\\#installation) Installation\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install hyperbrowser\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/extract\\#usage) Usage\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\nimport { z } from \"zod\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const schema = z.object({\n productName: z.string(),\n productOverview: z.string(),\n keyFeatures: z.array(z.string()),\n pricing: z.array(\n z.object({\n plan: z.string(),\n price: z.string(),\n features: z.array(z.string()),\n })\n ),\n });\n\n // Handles both starting and waiting for extract job response\n const result = await client.extract.startAndWait({\n urls: [\"https://hyperbrowser.ai\"],\n prompt:\n \"Extract the product name, an overview of the product, its key features, and a list of its pricing plans from the page.\",\n schema: schema,\n });\n\n console.log(\"result\", JSON.stringify(result, null, 2));\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nimport json\nfrom typing import List\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.extract import StartExtractJobParams\nfrom pydantic import BaseModel\n\n# Load environment variables from .env file\nload_dotenv()\n\n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nclass PricingSchema(BaseModel):\n plan: str\n price: str\n features: List[str]\n\nclass ExtractSchema(BaseModel):\n product_name: str\n product_overview: str\n key_features: List[str]\n pricing: List[PricingSchema]\n\ndef main():\n result = client.extract.start_and_wait(\n params=StartExtractJobParams(\n urls=[\"https://hyperbrowser.ai\"],\n prompt=\"Extract the product name, an overview of the product, its key features, and a list of its pricing plans from the page.\",\n schema=ExtractSchema,\n )\n )\n print(\"result:\", json.dumps(result.data, indent=2))\n\nmain()\n```\n\nStart Extract Job\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/extract \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"urls\": [\"https://hyperbrowser.ai\"],\n \"prompt\": \"Extract the product name, an overview of the product, its key features, and a list of its pricing plans from the page.\",\n \"schema\": {\n \"type\": \"object\",\n \"properties\": {\n \"productName\": {\n \"type\": \"string\"\n },\n \"productOverview\": {\n \"type\": \"string\"\n },\n \"keyFeatures\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"pricing\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"plan\": {\n \"type\": \"string\"\n },\n \"price\": {\n \"type\": \"string\"\n },\n \"features\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n },\n \"required\": [\\\n \"plan\",\\\n \"price\",\\\n \"features\"\\\n ]\n }\n }\n },\n \"required\": [\\\n \"productName\",\\\n \"productOverview\",\\\n \"keyFeatures\",\\\n \"pricing\"\\\n ]\n }\n }'\n```\n\nGet Extract Job Status and Data\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl https://app.hyperbrowser.ai/api/extract/{jobId} \\\n -H 'x-api-key: '\n```\n\nYou can configure the extract request with the following parameters:\n\n- `urls` \\- A required list of urls you want to use to extract data from. To allow crawling for any of the urls provided in the list, simply add `/*` to the end of the url ( `https://hyperbrowser.ai/*`). This will crawl other pages on the site with the same origin and find relevant pages to use for the extraction context.\n\n- `schema` \\- A strict json schema you want the returned data to be structured as. Gives the best results.\n\n- `prompt` \\- A prompt describing how you want the data structured. Useful if you don't have a specific schema in mind.\n\n- `maxLinks` \\- The maximum number of links to look for if performing a crawl for any given url.\n\n- `waitFor` \\- A delay in milliseconds to wait after the page loads before initiating the scrape to get data for extraction from page. This can be useful for allowing dynamic content to fully render. This is also useful for waiting to detect CAPTCHAs on the page if you have `solveCaptchas` set to true in the `sessionOptions`.\n\n- `sessionOptions` \\- [Options for the session](https://www.docs.hyperbrowser.ai/web-scraping/extract#session-configurations).\n\n\nYou must provide either a `schema` or a `prompt` in your request, and if both are provided the schema takes precedence.\n\nFor the Node SDK, you can simply pass in a zod schema for ease of use or an actual json schema. For the Python SDK, you can pass in a pydantic model or an actual json schema.\n\nEnsure that the root level of the schema is `type: \"object\"` .\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/extract\\#response) Response\n\nThe Start Extract Job `POST /extract` endpoint will return a `jobId` in the response which can be used to get information about the job in subsequent requests.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"962372c4-a140-400b-8c26-4ffe21d9fb9c\"\n}\n```\n\nThe Get Extract Job `GET /extract/{jobId}` will return the following data:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"962372c4-a140-400b-8c26-4ffe21d9fb9c\",\n \"status\": \"completed\",\n \"data\": {\n \"pricing\": [\\\n {\\\n \"plan\": \"Free\",\\\n \"price\": \"$0\",\\\n \"features\": [\\\n \"3,000 Credits Included\",\\\n \"5 Concurrent Browsers\",\\\n \"7 Days Data Retention\",\\\n \"Basic Stealth Mode\"\\\n ]\\\n },\\\n {\\\n \"plan\": \"Startup\",\\\n \"price\": \"$30 / Month\",\\\n \"features\": [\\\n \"18,000 Credits Included\",\\\n \"25 Concurrent Browsers\",\\\n \"30 Day Data Retention\",\\\n \"Auto Captcha Solving\",\\\n \"Basic Stealth Mode\"\\\n ]\\\n },\\\n {\\\n \"plan\": \"Scale\",\\\n \"price\": \"$100 / Month\",\\\n \"features\": [\\\n \"60,000 Credits Included\",\\\n \"100 Concurrent Browsers\",\\\n \"30 Day Data Retention\",\\\n \"Auto Captcha Solving\",\\\n \"Advanced Stealth Mode\"\\\n ]\\\n },\\\n {\\\n \"plan\": \"Enterprise\",\\\n \"price\": \"Custom\",\\\n \"features\": [\\\n \"Volume discounts available\",\\\n \"Premium Support\",\\\n \"HIPAA/SOC 2\",\\\n \"250+ Concurrent Browsers\",\\\n \"180+ Day Data Retention\",\\\n \"Auto Captcha Solving\",\\\n \"Advanced Stealth Mode\"\\\n ]\\\n }\\\n ],\n \"keyFeatures\": [\\\n \"Run headless browsers to automate tasks like web scraping, testing, and form filling.\",\\\n \"Use browsers to scrape and structure web data at scale for analysis and insights.\",\\\n \"Integrate with AI agents to enable browsing, data collection, and interaction with web apps.\",\\\n \"Automatically solve captchas to streamline automation workflows.\",\\\n \"Operate browsers in stealth mode to bypass bot detection and stay undetected.\",\\\n \"Manage browser sessions with logging, debugging, and secure resource isolation.\"\\\n ],\n \"productName\": \"Hyperbrowser\",\n \"productOverview\": \"Hyperbrowser is a platform for running and scaling headless browsers in secure, isolated containers. Built for web automation and AI-driven use cases.\"\n }\n}\n```\n\nThe status of an extract job can be one of `pending`, `running`, `completed`, `failed` . There can also be an optional `error` field with an error message if an error was encountered.\n\nTo see the full schema, checkout the [API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/extract).\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/extract\\#session-configurations) Session Configurations\n\nYou can also provide configurations for the session that will be used to execute the extract job just as you would when creating a new session itself. These could include using a proxy or solving CAPTCHAs. To see the full list of session configurations, checkout the [Session API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions).\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { config } from \"dotenv\";\nimport { z } from \"zod\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const schema = z.object({\n productName: z.string(),\n productOverview: z.string(),\n keyFeatures: z.array(z.string()),\n pricing: z.array(\n z.object({\n plan: z.string(),\n price: z.string(),\n features: z.array(z.string()),\n })\n ),\n });\n\n const result = await client.extract.startAndWait({\n urls: [\"https://hyperbrowser.ai\"],\n prompt:\n \"Extract the product name, an overview of the product, its key features, and its pricing plans from the page.\",\n schema: schema,\n // include sessionOptions\n sessionOptions: {\n useProxy: true,\n solveCaptchas: true,\n },\n });\n\n console.log(\"result\", JSON.stringify(result, null, 2));\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.extract import StartExtractJobParams\nfrom pydantic import BaseModel\n\nload_dotenv()\n\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nclass PricingSchema(BaseModel):\n plan: str\n price: str\n features: List[str]\n\nclass ExtractSchema(BaseModel):\n product_name: str\n product_overview: str\n key_features: List[str]\n pricing: List[PricingSchema]\n\ndef main():\n result = client.extract.start_and_wait(\n params=StartExtractJobParams(\n urls=[\"https://hyperbrowser.ai\"],\n prompt=\"Extract the product name, an overview of the product, its key features, and a list of its pricing plans from the page.\",\n schema=ExtractSchema,\n # include session_options\n session_options=CreateSessionParams(use_proxy=True, solve_captchas=True),\n )\n )\n print(\"result:\", json.dumps(result.data, indent=2))\n\nmain()\n```\n\nHyperbrowser's CAPTCHA solving and proxy usage features require being on a `PAID` plan.\n\nUsing proxy and solving CAPTCHAs will slow down the page scraping in the extract job so use it only if necessary.\n\nFor a full reference on the extract endpoint, checkout the [API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/extract).\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/web-scraping/extract\\#billing) Billing\n\nCredit usage for extract jobs are charged based on the total number of output tokens used for successful extract jobs. Each output token costs **0.015 credits** which comes out to $30 per million output tokens.\n\n[PreviousCrawl](https://www.docs.hyperbrowser.ai/web-scraping/crawl) [NextBrowser Use](https://www.docs.hyperbrowser.ai/agents/browser-use)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/web-scraping/extract", "title": "Extract | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/v5mRmWCbzPIVqSycmGUX", "og:title": "Extract | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/web-scraping/extract", "description": "Extract data from pages using AI", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/v5mRmWCbzPIVqSycmGUX", "twitter:title": "Extract | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/web-scraping/extract", "og:description": "Extract data from pages using AI", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Extract data from pages using AI" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/web-scraping/extract", "status": "completed", "error": null }, "summary": "The Extract page of Hyperbrowser provides guidelines for web scraping and data extraction techniques using the tool." }, { "pathname": "/agents/browser-use", "data": { "markdown": "Browser Use \\| Hyperbrowser\n\nBrowser Use is an open-source tool designed to make websites accessible for AI agents by enabling them to interact with web pages as a human user would. It provides a framework that allows AI systems to navigate, interpret, and manipulate web content, facilitating tasks such as data extraction, web automation, and testing.\n\nHyperbrowser's browser-use agent allows you to easily execute agent tasks on the web utilizing browser-use with just a simple call. Hyperbrowser exposes endpoints for starting/stopping a browser-use task and for getting it's status and results.\n\nBy default, browser-use tasks are handled in an asynchronous manner of first starting the task and then checking it's status until it is completed. However, if you don't want to handle the monitoring yourself, our SDKs provide a simple function that handles the whole flow and returns the data once the task is completed.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/browser-use\\#installation) Installation\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install hyperbrowser\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nuv add hyperbrowser\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/browser-use\\#usage) Usage\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst hbClient = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const result = await hbClient.agents.browserUse.startAndWait({\n task: \"go to Hacker News and summarize the top 5 posts of the day\",\n });\n\n console.log(`Output:\\n\\n${result.data?.finalResult}`);\n};\n\nmain().catch((err) => {\n console.error(`Error: ${err.message}`);\n});\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models import StartBrowserUseTaskParams\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nhb_client = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\ndef main():\n resp = hb_client.agents.browser_use.start_and_wait(\n StartBrowserUseTaskParams(\n task=\"go to Hacker News and summarize the top 5 posts of the day\"\n )\n )\n\n print(f\"Output:\\n\\n{resp.data.final_result}\")\n\nif __name__ == \"__main__\":\n try:\n main()\n except Exception as e:\n print(f\"Error: {e}\")\n```\n\nStart browser-use task\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/task/browser-use \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"task\": \"go to Hacker News and summarize the top 5 posts of the day\"\n }'\n```\n\nGet browser-use task status\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl https://app.hyperbrowser.ai/api/task/browser-use/{jobId}/status \\\n -H 'x-api-key: '\n```\n\nGet browser-use task\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl https://app.hyperbrowser.ai/api/task/browser-use/{jobId} \\\n -H 'x-api-key: '\n```\n\nStop browser-use task\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X PUT https://app.hyperbrowser.ai/api/task/browser-use/{jobId}/stop \\\n -H 'x-api-key: '\n```\n\nBrowser use tasks can be configured with a number of parameters. Some of them are described briefly here, but a list can be found in our [Browser Use API Reference.](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use)\n\nBrowser-Use Task parameters [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/browser-use#browser-use-task-parameters)\n\n- `llm` \\- The language model (LLM) instance to use for generating actions. By default, Hyperbrowser will use Gemini-2 Flash. A complete list is available in the [Browser Use API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use)\n\n\n- `sessionId` \\- An optional existing browser session ID to connect to instead of creating a new one.\n\n\n- `validateOutput` \\- When enabled, validates the agent's output format to ensure proper structure.\n\n\n- `useVision` \\- When enabled, allows the agent to analyze screenshots of the webpage for better context understanding.\n\n\n- `useVisionForPlanner` \\- When enabled, provides screenshots to the planning component of the agent.\n\n\n- `maxActionsPerStep` \\- The maximum number of actions the agent can perform in a single step before reassessing.\n\n\n- `maxInputTokens` \\- Maximum token limit for inputs sent to the language model, preventing oversized contexts.\n\n\n- `plannerLlm` \\- The language model to use specifically for planning future actions, can differ from the main LLM. By default, Hyperbrowser will use Gemini-2 Flash\n\n\n- `pageExtractionLlm` \\- The language model to use for extracting structured data from webpages. By default, Hyperbrowser will use Gemini-2 Flash\n\n\n- `plannerInterval` \\- How often the planner runs (measured in agent steps) to reassess the overall strategy.\n\n\n- `maxSteps` \\- The maximum number of steps the agent can take before concluding the task.\n\n\n- `keepBrowserOpen` \\- When enabled, keeps the browser session open after task completion.\n\n\n- `sessionOptions` \\- [Options for the session](https://www.docs.hyperbrowser.ai/agents/browser-use#session-configurations).\n\n\nFor detailed usage/schema, check out the [Browser Use API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use).\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/browser-use\\#session-configurations) Session Configurations\n\nYou can also provide configurations for the session that will be used to execute the browser-use task just as you would when creating a new session itself. These could include using a proxy or solving CAPTCHAs. To see the full list of session configurations, checkout the [Session API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions).\n\nThe `sessionOptions` will only apply if creating a new session when no `sessionId` is provided.\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst hbClient = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const result = await hbClient.agents.browserUse.startAndWait({\n task: \"go to Hacker News and summarize the top 5 posts of the day\",\n sessionOptions: {\n acceptCookies: true,\n }\n });\n\n console.log(`Output:\\n\\n${result.data?.finalResult}`);\n};\n\nmain().catch((err) => {\n console.error(`Error: ${err.message}`);\n});\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models import StartBrowserUseTaskParams, CreateSessionParams\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nhb_client = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\ndef main():\n resp = hb_client.agents.browser_use.start_and_wait(\n StartBrowserUseTaskParams(\n task=\"go to Hacker News and summarize the top 5 posts of the day\",\n session_options=CreateSessionParams(\n accept_cookies=True,\n ),\n )\n )\n\n print(f\"Output:\\n\\n{resp.data.final_result}\")\n\nif __name__ == \"__main__\":\n try:\n main()\n except Exception as e:\n print(f\"Error: {e}\")\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/task/browser-use \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"task\": \"go to Hacker News and summarize the top 5 posts of the day\",\n \"sessionOptions\": {\n \"acceptCookies\": true\n }\n }'\n```\n\nHyperbrowser's CAPTCHA solving and proxy usage features require being on a `PAID` plan.\n\nUsing proxy and solving CAPTCHAs will slow down the web navigation in the browser-use task so use it only if necessary.\n\n[PreviousExtract](https://www.docs.hyperbrowser.ai/web-scraping/extract) [NextClaude Computer Use](https://www.docs.hyperbrowser.ai/agents/claude-computer-use)\n\nLast updated 15 hours ago", "metadata": { "url": "https://docs.hyperbrowser.ai/agents/browser-use", "title": "Browser Use | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/CrH0SK49GtKcR5ztoqwJ", "og:title": "Browser Use | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/agents/browser-use", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/CrH0SK49GtKcR5ztoqwJ", "twitter:title": "Browser Use | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/agents/browser-use", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/agents/browser-use", "status": "completed", "error": null }, "summary": "The page discusses using Hyperbrowser for web scraping and automation tasks via browser interactions." }, { "pathname": "/agents/claude-computer-use", "data": { "markdown": "Claude Computer Use \\| Hyperbrowser\n\nClaude's Computer Use allows Claude to directly interact with your computer to perform tasks much like a human. This capability allows Claude to move the cursor, click buttons, type text, and navigate the web, thereby automating complex, multi-step workflows.\n\nHyperbrowser's Claude Computer Use agent allows you to easily execute agent tasks on the web with just a simple call. Hyperbrowser exposes endpoints for starting/stopping a Claude Computer Use task and for getting it's status and results.\n\nBy default, these tasks are handled in an asynchronous manner of first starting the task and then checking it's status until it is completed. However, if you don't want to handle the monitoring yourself, our SDKs provide a simple function that handles the whole flow and returns the data once the task is completed.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/claude-computer-use\\#installation) Installation\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install hyperbrowser\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nuv add hyperbrowser\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/claude-computer-use\\#usage) Usage\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst hbClient = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const result = await hbClient.agents.claudeComputerUse.startAndWait({\n task: \"what are the top 5 posts on Hacker News\",\n });\n\n console.log(`Output:\\n\\n${result.data?.finalResult}`);\n};\n\nmain().catch((err) => {\n console.error(`Error: ${err.message}`);\n});\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models import StartClaudeComputerUseTaskParams\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nhb_client = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\ndef main():\n resp = hb_client.agents.claude_computer_use.start_and_wait(\n StartClaudeComputerUseTaskParams(\n task=\"what are the top 5 posts on Hacker News\"\n )\n )\n\n print(f\"Output:\\n\\n{resp.data.final_result}\")\n\nif __name__ == \"__main__\":\n try:\n main()\n except Exception as e:\n print(f\"Error: {e}\")\n```\n\nStart Claude Computer Use task\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/task/claude-computer-use \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"task\": \"what are the top 5 posts on Hacker News\"\n }'\n```\n\nGet Claude Computer Use task status\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl https://app.hyperbrowser.ai/api/task/claude-computer-use/{jobId}/status \\\n -H 'x-api-key: '\n```\n\nGet Claude Computer Use task\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl https://app.hyperbrowser.ai/api/task/claude-computer-use/{jobId} \\\n -H 'x-api-key: '\n```\n\nStop Claude Computer Use task\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X PUT https://app.hyperbrowser.ai/api/task/claude-computer-use/{jobId}/stop \\\n -H 'x-api-key: '\n```\n\nClaude Computer Use tasks can be configured with a number of parameters. Some of them are described briefly here, but a list can be found in our [Claude Computer Use API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use).\n\nTask parameters [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/claude-computer-use#task-parameters)\n\n- `task` \\- The instruction or goal to be accomplished by Claude Computer Use.\n\n\n- `sessionId` \\- An optional existing browser session ID to connect to instead of creating a new one.\n\n\n- `maxFailures` \\- The maximum number of consecutive failures allowed before the task is aborted.\n\n\n- `maxSteps` \\- The maximum number of interaction steps the agent can take to complete the task.\n\n\n- `keepBrowserOpen` \\- When enabled, keeps the browser session open after task completion.\n\n\n- `sessionOptions` \\- [Options for the session](https://www.docs.hyperbrowser.ai/agents/claude-computer-use#session-configurations).\n\n\nFor detailed usage/schema, check out the [Claude Computer Use API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use).\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/claude-computer-use\\#session-configurations) Session Configurations\n\nYou can also provide configurations for the session that will be used to execute the task just as you would when creating a new session itself. These could include using a proxy or solving CAPTCHAs. To see the full list of session configurations, checkout the [Session API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions).\n\nThe `sessionOptions` will only apply if creating a new session when no `sessionId` is provided.\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst hbClient = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const result = await hbClient.agents.claudeComputerUse.startAndWait({\n task: \"what are the top 5 posts on Hacker News\",\n sessionOptions: {\n acceptCookies: true,\n }\n });\n\n console.log(`Output:\\n\\n${result.data?.finalResult}`);\n};\n\nmain().catch((err) => {\n console.error(`Error: ${err.message}`);\n});\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models import StartClaudeComputerUseTaskParams, CreateSessionParams\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nhb_client = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\ndef main():\n resp = hb_client.agents.claude_computer_use.start_and_wait(\n StartClaudeComputerUseTaskParams(\n task=\"what are the top 5 posts on Hacker News\",\n session_options=CreateSessionParams(\n accept_cookies=True,\n ),\n )\n )\n\n print(f\"Output:\\n\\n{resp.data.final_result}\")\n\nif __name__ == \"__main__\":\n try:\n main()\n except Exception as e:\n print(f\"Error: {e}\")\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/task/claude-computer-use \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"task\": \"what are the top 5 posts on Hacker News\",\n \"sessionOptions\": {\n \"acceptCookies\": true\n }\n }'\n```\n\nHyperbrowser's CAPTCHA solving and proxy usage features require being on a `PAID` plan.\n\nUsing proxy and solving CAPTCHAs will slow down the web navigation in the Claude Computer Use task so use it only if necessary.\n\n[PreviousBrowser Use](https://www.docs.hyperbrowser.ai/agents/browser-use) [NextOpenAI CUA](https://www.docs.hyperbrowser.ai/agents/openai-cua)\n\nLast updated 15 hours ago", "metadata": { "url": "https://docs.hyperbrowser.ai/agents/claude-computer-use", "title": "Claude Computer Use | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/JMaUzHUSLezFYgVTb6f8", "og:title": "Claude Computer Use | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/agents/claude-computer-use", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/JMaUzHUSLezFYgVTb6f8", "twitter:title": "Claude Computer Use | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/agents/claude-computer-use", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/agents/claude-computer-use", "status": "completed", "error": null }, "summary": "The page provides guidelines on using Claude with Hyperbrowser for effective web scraping and automation." }, { "pathname": "/agents/openai-cua", "data": { "markdown": "OpenAI CUA \\| Hyperbrowser\n\n鈥婳penAI's Computer-Using Agent (CUA) is an advanced AI model designed to interact with computer interfaces much like a human user. It can navigate graphical user interfaces (GUIs), perform tasks such as clicking buttons, typing text, and managing multi-step processes. This capability allows CUA to automate complex workflows without relying on specialized APIs. Actually, this is the model that powers Operator, OpenAI's AI agent capable of executing various web-based tasks like filling out forms, ordering products, and scheduling appointments. Operator utilizes CUA to mimic human interactions within a browser, allowing it to handle repetitive or intricate tasks on behalf of users.\n\nHyperbrowser's CUA agent allows you to easily execute agent tasks on the web utilizing OpenAI's Computer-Using Agent with just a simple call. Hyperbrowser exposes endpoints for starting/stopping a CUA task and for getting it's status and results.\n\nBy default, CUA tasks are handled in an asynchronous manner of first starting the task and then checking it's status until it is completed. However, if you don't want to handle the monitoring yourself, our SDKs provide a simple function that handles the whole flow and returns the data once the task is completed.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/openai-cua\\#installation) Installation\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add @hyperbrowser/sdk\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install hyperbrowser\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nuv add hyperbrowser\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/openai-cua\\#usage) Usage\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst hbClient = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const result = await hbClient.agents.cua.startAndWait({\n task: \"what are the top 5 posts on Hacker News\",\n });\n\n console.log(`Output:\\n\\n${result.data?.finalResult}`);\n};\n\nmain().catch((err) => {\n console.error(`Error: ${err.message}`);\n});\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models import StartCuaTaskParams\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nhb_client = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\ndef main():\n resp = hb_client.agents.cua.start_and_wait(\n StartCuaTaskParams(\n task=\"what are the top 5 posts on Hacker News\"\n )\n )\n\n print(f\"Output:\\n\\n{resp.data.final_result}\")\n\nif __name__ == \"__main__\":\n try:\n main()\n except Exception as e:\n print(f\"Error: {e}\")\n```\n\nStart CUA task\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/task/cua \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"task\": \"what are the top 5 posts on Hacker News\"\n }'\n```\n\nGet CUA task status\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl https://app.hyperbrowser.ai/api/task/cua/{jobId}/status \\\n -H 'x-api-key: '\n```\n\nGet CUA task\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl https://app.hyperbrowser.ai/api/task/cua/{jobId} \\\n -H 'x-api-key: '\n```\n\nStop CUA task\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X PUT https://app.hyperbrowser.ai/api/task/cua/{jobId}/stop \\\n -H 'x-api-key: '\n```\n\nCUA tasks can be configured with a number of parameters. Some of them are described briefly here, but a list can be found in our [OpenAI CUA API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua).\n\nTask parameters [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/openai-cua#task-parameters)\n\n- `task` \\- The instruction or goal to be accomplished by CUA.\n\n\n- `sessionId` \\- An optional existing browser session ID to connect to instead of creating a new one.\n\n\n- `maxFailures` \\- The maximum number of consecutive failures allowed before the task is aborted.\n\n\n- `maxSteps` \\- The maximum number of interaction steps CUA can take to complete the task.\n\n\n- `keepBrowserOpen` \\- When enabled, keeps the browser session open after task completion.\n\n\n- `sessionOptions` \\- [Options for the session](https://www.docs.hyperbrowser.ai/agents/openai-cua#session-configurations).\n\n\nFor detailed usage/schema, check out the [OpenAI CUA API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua).\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/agents/openai-cua\\#session-configurations) Session Configurations\n\nYou can also provide configurations for the session that will be used to execute the CUA task just as you would when creating a new session itself. These could include using a proxy or solving CAPTCHAs. To see the full list of session configurations, checkout the [Session API Reference](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions).\n\nThe `sessionOptions` will only apply if creating a new session when no `sessionId` is provided.\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst hbClient = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const result = await hbClient.agents.cua.startAndWait({\n task: \"what are the top 5 posts on Hacker News\",\n sessionOptions: {\n acceptCookies: true,\n }\n });\n\n console.log(`Output:\\n\\n${result.data?.finalResult}`);\n};\n\nmain().catch((err) => {\n console.error(`Error: ${err.message}`);\n});\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models import StartCuaTaskParams, CreateSessionParams\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nhb_client = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\ndef main():\n resp = hb_client.agents.cua.start_and_wait(\n StartCuaTaskParams(\n task=\"what are the top 5 posts on Hacker News\",\n session_options=CreateSessionParams(\n accept_cookies=True,\n ),\n )\n )\n\n print(f\"Output:\\n\\n{resp.data.final_result}\")\n\nif __name__ == \"__main__\":\n try:\n main()\n except Exception as e:\n print(f\"Error: {e}\")\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/task/cua \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"task\": \"what are the top 5 posts on Hacker News\",\n \"sessionOptions\": {\n \"acceptCookies\": true\n }\n }'\n```\n\nHyperbrowser's CAPTCHA solving and proxy usage features require being on a `PAID` plan.\n\nUsing proxy and solving CAPTCHAs will slow down the web navigation in the CUA task so use it only if necessary.\n\n[PreviousClaude Computer Use](https://www.docs.hyperbrowser.ai/agents/claude-computer-use) [NextScraping](https://www.docs.hyperbrowser.ai/guides/scraping)\n\nLast updated 15 hours ago", "metadata": { "url": "https://docs.hyperbrowser.ai/agents/openai-cua", "title": "OpenAI CUA | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/NIlWDGVk2CmVfc86sNFl", "og:title": "OpenAI CUA | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/agents/openai-cua", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/NIlWDGVk2CmVfc86sNFl", "twitter:title": "OpenAI CUA | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/agents/openai-cua", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/agents/openai-cua", "status": "completed", "error": null }, "summary": "The page discusses the integration of OpenAI's CUA with Hyperbrowser for enhanced web scraping and automation capabilities." }, { "pathname": "/guides/ai-function-calling", "data": { "markdown": "AI Function Calling \\| Hyperbrowser\n\nHyperbrowser integrates seamlessly with OpenAI and Anthropic's function calling APIs, enabling you to enhance your AI applications with web scraping and crawling capabilities. This guide will walk you through setting up and using Hyperbrowser's scrape and crawl tools with OpenAI and Anthropic.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/ai-function-calling\\#setup) Setup\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/ai-function-calling\\#installation) Installation\n\nFirst, install the necessary dependencies to run our script.\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk dotenv openai\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install hyperbrowser openai python-dotenv\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/ai-function-calling\\#setup-your-environment) Setup your Environment\n\nTo use Hyperbrowser with your code, you will need an API Key. You can get one easily from the [dashboard](https://app.hyperbrowser.ai/settings?tab=api_key). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY` . You will also need an `OPENAI_API_KEY`.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/ai-function-calling\\#code) Code\n\nNodePython\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport OpenAI from \"openai\";\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { WebsiteCrawlTool, WebsiteScrapeTool,WebsiteExtractTool } from \"@hyperbrowser/sdk/tools\";\nimport { config } from \"dotenv\";\n\nconfig();\n\n// Initialize clients\nconst hb = new Hyperbrowser({ apiKey: process.env.HYPERBROWSER_API_KEY });\nconst oai = new OpenAI({\n apiKey: process.env.OPENAI_API_KEY,\n});\n\nasync function handleToolCall(tc) {\n console.log(\"Handling tool call\");\n\n try {\n const args = JSON.parse(tc.function.arguments);\n console.log(`Tool call ID: ${tc.id}`);\n console.log(`Function name: ${tc.function.name}`);\n console.log(`Function args: ${JSON.stringify(args, null, 2)}`);\n console.log(\"-\".repeat(50));\n\n if (\n tc.function.name === WebsiteCrawlTool.openaiToolDefinition.function.name\n ) {\n const response = await WebsiteCrawlTool.runnable(hb, args);\n return {\n tool_call_id: tc.id,\n content: response,\n role: \"tool\",\n };\n } else if (\n tc.function.name === WebsiteScrapeTool.openaiToolDefinition.function.name\n ) {\n const response = await WebsiteScrapeTool.runnable(hb, args);\n return {\n tool_call_id: tc.id,\n content: response,\n role: \"tool\",\n };\n } else if (\n tc.function.name === WebsiteExtractTool.openaiToolDefinition.function.name\n ) {\n const response = await WebsiteExtractTool.runnable(hb, args);\n return {\n tool_call_id: tc.id,\n content: response,\n role: \"tool\",\n };\n } else {\n return {\n tool_call_id: tc.id,\n content: \"Unknown tool call\",\n role: \"tool\",\n };\n }\n } catch (e) {\n console.error(e);\n return {\n role: \"tool\",\n tool_call_id: tc.id,\n content: `Error occurred: ${e}`,\n };\n }\n}\n\nconst messages = [\\\n {\\\n role: \"user\",\\\n content: \"What does Hyperbrowser.ai do? Provide citations.\",\\\n },\\\n];\n\nasync function openaiChat() {\n while (true) {\n const resp = await oai.beta.chat.completions.parse({\n model: \"gpt-4o-mini\",\n messages: messages,\n tools: [\\\n WebsiteCrawlTool.openaiToolDefinition,\\\n WebsiteScrapeTool.openaiToolDefinition,\\\n WebsiteExtractTool.openaiToolDefinition,\\\n ],\n });\n\n const choice = resp.choices[0];\n messages.push(choice.message);\n\n if (choice.finish_reason === \"tool_calls\") {\n for (const tc of choice.message.tool_calls) {\n const result = await handleToolCall(tc);\n messages.push(result);\n }\n } else if (choice.finish_reason === \"stop\") {\n console.log(choice.message.content);\n break;\n } else {\n throw new Error(\"Unknown Error Occurred\");\n }\n }\n}\n\nopenaiChat();\n\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport json\nimport os\n\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.tools import WebsiteCrawlTool, WebsiteScrapeTool, WebsiteExtractTool\n\nfrom openai import OpenAI\nfrom openai.types.chat import (\n ChatCompletionMessageToolCall,\n ChatCompletionMessageParam,\n ChatCompletionToolMessageParam,\n)\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\noai = OpenAI(api_key=os.getenv(\"OPENAI_API_KEY\"))\nhb = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\ndef handle_tool_call(\n tc: ChatCompletionMessageToolCall,\n) -> ChatCompletionToolMessageParam:\n print(\"Handling tool call\")\n\n try:\n args = json.loads(tc.function.arguments)\n print(f\"Tool call ID: {tc.id}\")\n print(f\"Function name: {tc.function.name}\")\n print(f\"Function args: {args}\")\n print(\"-\" * 50)\n if (\n tc.function.name\n == WebsiteCrawlTool.openai_tool_definition[\"function\"][\"name\"]\n ):\n response = WebsiteCrawlTool.runnable(hb, args)\n return {\"tool_call_id\": tc.id, \"content\": response, \"role\": \"tool\"}\n elif (\n tc.function.name\n == WebsiteScrapeTool.openai_tool_definition[\"function\"][\"name\"]\n ):\n response = WebsiteScrapeTool.runnable(hb, args)\n return {\"tool_call_id\": tc.id, \"content\": response, \"role\": \"tool\"}\n elif (\n tc.function.name\n == WebsiteExtractTool.openai_tool_definition[\"function\"][\"name\"]\n ):\n response = WebsiteExtractTool.runnable(hb, args)\n return {\"tool_call_id\": tc.id, \"content\": response, \"role\": \"tool\"}\n else:\n return {\n \"tool_call_id\": tc.id,\n \"content\": \"Unknown tool call\",\n \"role\": \"tool\",\n }\n except Exception as e:\n print(e)\n return {\"role\": \"tool\", \"tool_call_id\": tc.id, \"content\": f\"Error occured: {e}\"}\n\nmessages: list[ChatCompletionMessageParam] = [\\\n {\\\n \"role\": \"user\",\\\n \"content\": \"What does Hyperbrowser.ai do? Provide citations.\",\\\n },\\\n]\n\nwhile True:\n response = oai.beta.chat.completions.parse(\n messages=messages,\n model=\"gpt-4o-mini\",\n tools=[\\\n WebsiteCrawlTool.openai_tool_definition,\\\n WebsiteScrapeTool.openai_tool_definition,\\\n WebsiteExtractTool.openai_tool_definition,\\\n ],\n )\n\n choice = response.choices[0]\n messages.append(choice.message) # type: ignore\n if choice.finish_reason == \"tool_calls\":\n for tc in choice.message.tool_calls: # type: ignore\n result = handle_tool_call(tc=tc)\n messages.append(result)\n\n elif choice.finish_reason == \"stop\":\n print(choice.message.content)\n break\n\n else:\n raise Exception(\"Unknown Error Occured\")\n\n```\n\nHyperbrowser exposes a `WebsiteCrawlTool`, a `WebsiteScrapeTool`, and a `WebsiteExtractTool` to be used for OpenAI and Anthropic function calling. Each class provides a tool definition for both OpenAI and Anthropic that defines the schema which can be passed into the tools argument. Then, in the `handleToolCall` function, we parse the tool call arguments and dispatch the appropriate tool class `runnable` function based on the function name which will return the result of the scrape or crawl in formatted markdown.\n\n[PreviousScraping](https://www.docs.hyperbrowser.ai/guides/scraping) [NextExtract Information with an LLM](https://www.docs.hyperbrowser.ai/guides/extract-information-with-an-llm)\n\nLast updated 16 hours ago", "metadata": { "url": "https://docs.hyperbrowser.ai/guides/ai-function-calling", "title": "AI Function Calling | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/HOPnYKVapgTqKmbTj9Px", "og:title": "AI Function Calling | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/guides/ai-function-calling", "description": "Using Hyperbrowser with OpenAI and Anthropic Function Tools", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/HOPnYKVapgTqKmbTj9Px", "twitter:title": "AI Function Calling | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/guides/ai-function-calling", "og:description": "Using Hyperbrowser with OpenAI and Anthropic Function Tools", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Using Hyperbrowser with OpenAI and Anthropic Function Tools" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/guides/ai-function-calling", "status": "completed", "error": null }, "summary": "The page discusses AI function calling features within Hyperbrowser for enhanced web scraping and automation." }, { "pathname": "/guides/scraping", "data": { "markdown": "Scraping \\| Hyperbrowser\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#scraping-a-web-page) Scraping a web page\n\nWith supplying just a url, you can easily extract the contents of a page in markdown format with the `/scrape` endpoint.\n\nNodePythoncURL\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n // Handles both starting and waiting for scrape job response\n const scrapeResult = await client.scrape.startAndWait({\n url: \"https://example.com\",\n });\n console.log(\"Scrape result:\", scrapeResult);\n};\n\nmain();\n```\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom hyperbrowser import Hyperbrowser\nfrom hyperbrowser.models.scrape import StartScrapeJobParams\n鈥媆n# Load environment variables from .env file\nload_dotenv()\n鈥媆n# Initialize Hyperbrowser client\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n鈥媆n鈥媆n# Start scraping and wait for completion\nscrape_result = client.scrape.start_and_wait(\n StartScrapeJobParams(url=\"https://example.com\")\n)\nprint(\"Scrape result:\", scrape_result)\n```\n\nStart Scrape Job\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/scrape \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"url\": \"https://example.com\"\n }'\n```\n\nGet Scrape Job Status and Data\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl https://app.hyperbrowser.ai/api/scrape/{jobId} \\\n -H 'x-api-key: '\n```\n\nNow, let's take an in depth look at all the provided options for scraping.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#session-options) Session Options\n\nAll Scraping APIs, like\n\n- Scrape\n\n- Crawl, and\n\n- Extract\n\n\nSupport the session parameters. You can see the [session parameters listed here](https://www.docs.hyperbrowser.ai/sessions/overview/session-parameters).\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#scrape-options) Scrape Options\n\n`formats`\n\n- **Type**: `array`\n\n- **Items**: `string`\n\n- **Enum**: `[\"html\", \"links\", \"markdown\", \"screenshot\"]`\n\n- **Description**: Choose the formats to include in the API response:\n\n\n\n- `html` \\- Returns the scraped content as HTML.\n\n- `links` \\- Includes a list of links found on the page.\n\n- `markdown` \\- Provides the content in Markdown format.\n\n- `screenshot` \\- Provides a screenshot of the page.\n\n\n- **Default**: `[\"markdown\"]`\n\n\n`includeTags`\n\n- **Type**: `array`\n\n- **Items**: `string`\n\n- **Description**: Provide an array of HTML tags, classes, or IDs to include in the scraped content. Only elements matching these selectors will be returned.\n\n- **Default**: `undefined`\n\n\n`excludeTags`\n\n- **Type**: `array`\n\n- **Items**: `string`\n\n- **Description**: Provide an array of HTML tags, classes, or IDs to exclude from the scraped content. Elements matching these selectors will be omitted from the response.\n\n- **Default**: `undefined`\n\n\n`onlyMainContent`\n\n- **Type**: `boolean`\n\n- **Description**: When set to `true` (default), the API will attempt to return only the main content of the page, excluding common elements like headers, navigation menus, and footers. Set to `false` to return the full page content.\n\n- **Default**: `true`\n\n\n`waitFor`\n\n- **Type**: `number`\n\n- **Description**: Specify a delay in milliseconds to wait after the page loads before initiating the scrape. This can be useful for allowing dynamic content to fully render. This is also useful for waiting to detect CAPTCHAs on the page if you have `solveCaptchas` set to true in the `sessionOptions`.\n\n- **Default**: `0`\n\n\n`timeout`\n\n- **Type**: `number`\n\n- **Description**: Specify the maximum time in milliseconds to wait for the page to load before timing out. This would be like doing:\n\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nawait page.goto(\"https://example.com\", { waitUntil: \"load\", timeout: 30000 })\n```\n\n- **Default**: `30000` (30 seconds)\n\n\n`waitUntil`\n\n- **Type**: `string`\n\n- **Enum**: `[\"load\", \"domcontentloaded\", \"networkidle\"]`\n\n- **Description**: Specify the condition to wait for the page to load:\n\n\n\n- `domcontentloaded`: Wait until the HTML is fully parsed and DOM is ready\n\n- `load` \\- Wait until DOM and all resources are completely loaded\n\n- `networkidle` \\- Wait until no more network requests occur for a certain period of time\n\n\n- **Default**: `load`\n\n\n`screenshotOptions`\n\n- **Type**: `object`\n\n- **Properties**:\n\n\n\n- **fullPage** \\- Take screenshot of the full page beyond the viewport\n\n\n\n- **Type**: `boolean`\n\n- **Default**: `false`\n\n\n- **format** \\- The image type of the screenshot\n\n\n\n- **Type**: `string`\n\n- **Enum**: `[\"webp\", \"jpeg\", \"png\"]`\n\n- **Default**: `webp`\n\n\n- **Description**: Configurations for the returned screenshot. Only applicable if `screenshot` is provided in the `formats` array.\n\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#example) Example\n\nBy configuring these options when making a scrape request, you can control the format and content of the scraped data, as well as the behavior of the scraper itself.\n\nFor example, to scrape a page with the following:\n\n- In stealth mode\n\n- With CAPTCHA solving\n\n- Return only the main content as HTML\n\n- Exclude any `` elements\n\n- Wait 2 seconds after the page loads and before scraping\n\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/scrape \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"url\": \"https://example.com\",\n \"sessionOptions\": {\n \"useStealth\": true,\n \"solveCaptchas\": true\n },\n \"scrapeOptions\": {\n \"formats\": [\"html\"],\n \"onlyMainContent\": true,\n \"excludeTags\": [\"span\"],\n \"waitFor\": 2000\n }\n }'\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#crawl-a-site) Crawl a Site\n\nInstead of just scraping a single page, you might want to get all the content across multiple pages on a site. The `/crawl` endpoint is perfect for such a task. You can use the same `sessionOptions` and `scrapeOptions` as before for this endpoint as well. The crawl endpoint does have some extra parameters that are used to tailor the crawl to your scraping needs.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#crawl-options) Crawl Options\n\n**Limiting the Number of Pages to Crawl with** `maxPages`\n\n- **Type**: `integer`\n\n- **Minimum**: 1\n\n- **Description**: The maximum number of pages to crawl before stopping.\n\n\n**Following Links with** `followLinks`\n\n- **Type**: `boolean`\n\n- **Default**: `true`\n\n- **Description**: When set to `true`, the crawler will follow links found on the pages it visits, allowing it to discover new pages and expand the scope of the crawl. When set to `false`, the crawler will only visit the starting URL and any explicitly specified pages, without following any additional links.\n\n\n**Ignoring the Sitemap with** `ignoreSitemap`\n\n- **Type**: `boolean`\n\n- **Default**: `false`\n\n- **Description**: When set to `true`, the crawler will not pre-generate a list of urls from potential sitemaps it finds. The crawler will try to locate sitemaps beginning at the base URL of the URL provided in the `url` param.\n\n\n**Excluding Pages with** `excludePatterns`\n\n- **Type**: `array`\n\n- **Items**: `string`\n\n- **Description**: An array of regular expressions or wildcard patterns specifying which URLs should be excluded from the crawl. Any pages whose URLs' path match one of these patterns will be skipped.\n\n\n**Including Pages with** `includePatterns`\n\n- **Type**: `array`\n\n- **Items**: `string`\n\n- **Description**: An array of regular expressions or wildcard patterns specifying which URLs should be included in the crawl. Only pages whose URLs' path match one of these path patterns will be visited.\n\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#example-1) Example\n\nBy configuring these options when initiating a crawl, you can control the scope and behavior of the crawler to suit your specific needs.\n\nFor example, to crawl a site with the following:\n\n- Maximum of 5 pages\n\n- Only include `/blog` pages\n\n- Return only the main content as HTML\n\n- Exclude any `` elements\n\n- Wait 2 seconds after the page loads and before scraping\n\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/crawl \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"url\": \"https://example.com\",\n \"maxPages\": 5,\n \"includePatterns\": [\"/blog/*\"],\n \"scrapeOptions\": {\n \"formats\": [\"html\"],\n \"onlyMainContent\": true,\n \"excludeTags\": [\"span\"],\n \"waitFor\": 2000\n }\n }'\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#structured-extraction) Structured Extraction\n\nThe Extract API allows you to fetch data in a well-defined structure from any webpage or website with just a few lines of code. You can provide a list of web pages, and hyperbrowser will collate all the information together and extract the information that best fits the provided schema (or prompt). You have access to the same `SessionOptions` available here as well.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#extract-options) Extract Options:\n\n**Specifying all page to collect data from with** `urls`\n\n- **Type**: `array`\n\n- **Items**: `string`\n\n- **Required**: Yes\n\n- **Description**: List of URLs to extract data from. To crawl a site, add `/*` to a URL (e.g., `https://example.com/*`). This will crawl other pages on the site with the same origin and find relevant pages to use for the extraction context.\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#specify-the-extraction-schema) Specify the extraction `schema`\n\n- **Type**: `object`\n\n- **Required** `: ` No\n\n- **Description**: JSON schema defining the structure of the data you want to extract. Gives the best results with clear data structure requirements.\n\n- **Note**: You must provide either a `schema` or a `prompt`. If both are provided, the schema takes precedence.\n\n\n**Specify the data to be extracted from a** `prompt`\n\n- **Type**: `string`\n\n- **Required**: No\n\n- **Description**: A prompt describing how you want the data structured. Useful if you don't have a specific schema in mind.\n\n- **Note**: You must provide either a `schema` or a `prompt`. If both are provided, the schema takes precedence.\n\n- Default: undefined\n\n\n**Further specify the extraction process with a** `systemPrompt`\n\n- **Type**: `string`\n\n- **Required**: No\n\n- **Description**: Additional instructions for the extraction process to guide the AI's behavior.\n\n- Default: undefined\n\n\n**Specify the number of pages to collect information from with** `maxLinks`\n\n- **Type**: `number`\n\n- **Description**: Maximum number of links to follow when crawling a site for any given URL with `/*` suffix.\n\n- **Default**: `undefined`\n\n\n**Max time to wait on a page before extraction using** `waitFor`\n\n- **Type**: `number`\n\n- **Description**: Time in milliseconds to wait after page load before extraction. This can be useful for allowing dynamic content to fully render or for waiting to detect CAPTCHAs if you have `solveCaptchas` set to true.\n\n- **Default**: `0`\n\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/scraping\\#example-2) Example\n\nBy configuring these options when initiating a structured extraction, you can control the scope and behavior to suit your specific needs.\n\nFor example, to crawl a site with the following:\n\n- Maximum of 5 pages\n\n- Include `/products` on example.com, and as many subsequent pages as possible on test.com\n\n- Return the extracted data in the specified schema.\n\n- Wait 2 seconds after the page loads and before scraping\n\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -X POST https://app.hyperbrowser.ai/api/extract \\\n -H 'Content-Type: application/json' \\\n -H 'x-api-key: ' \\\n -d '{\n \"urls\": [\"https://example.com/products\",\"https://www.test.com/*\"],\n \"prompt\": \"Extract the product information from this page\",\n \"schema\": {\n \"type\": \"object\",\n \"properties\": {\n \"productName\": {\n \"type\": \"string\"\n },\n \"price\": {\n \"type\": \"string\"\n },\n \"features\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n },\n \"required\": [\\\n \"productName\",\\\n \"price\",\\\n \"features\"\\\n ]\n },\n \"maxLinks\": 5,\n \"waitFor\": 2000,\n \"sessionOptions\": {\n \"useStealth\": true,\n \"solveCaptchas\": true,\n \"adblock\": true\n }\n }'\n```\n\n[PreviousOpenAI CUA](https://www.docs.hyperbrowser.ai/agents/openai-cua) [NextAI Function Calling](https://www.docs.hyperbrowser.ai/guides/ai-function-calling)\n\nLast updated 7 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/guides/scraping", "title": "Scraping | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/tWGDgga0NbBE9gd9fbMv", "og:title": "Scraping | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/guides/scraping", "description": "Advanced Options for Hyperbrowser Scraping", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/tWGDgga0NbBE9gd9fbMv", "twitter:title": "Scraping | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/guides/scraping", "og:description": "Advanced Options for Hyperbrowser Scraping", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Advanced Options for Hyperbrowser Scraping" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/guides/scraping", "status": "completed", "error": null }, "summary": "The page covers web scraping techniques and documentation for using Hyperbrowser effectively." }, { "pathname": "/guides/extract-information-with-an-llm", "data": { "markdown": "Extract Information with an LLM \\| Hyperbrowser\n\nIn this guide, we'll use Hyperbrowser's Node.js SDK to get formatted data from a Wikipedia page and then feed it into an LLM like ChatGPT to extract the information we want. Our goal is to get a list of the most populous cities.\n\nFor most use cases, we would recommend using the [extract API](https://www.docs.hyperbrowser.ai/web-scraping/extract) itself.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/extract-information-with-an-llm\\#setup) Setup\n\nFirst, lets create a new Node.js project.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nmkdir wiki-scraper && cd wiki-scraper\nnpm init -y\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/extract-information-with-an-llm\\#installation) Installation\n\nNext, let's install the necessary dependencies to run our script.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk dotenv openai zod\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/extract-information-with-an-llm\\#setup-your-environment) Setup your Environment\n\nTo use Hyperbrowser with your code, you will need an API Key. You can get one easily from the [dashboard](https://app.hyperbrowser.ai/settings?tab=api_key). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY` . You will also need an `OPENAI_API_KEY` to use ChatGPT to extract information from our scraped data.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/extract-information-with-an-llm\\#code) Code\n\nNext, create a new file `scraper.js` and add the following code:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\nimport { z } from \"zod\";\nimport OpenAI from \"openai\";\nimport { zodResponseFormat } from \"openai/helpers/zod\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\nconst openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });\n\nconst CitySchema = z.object({\n city: z.string(),\n country: z.string(),\n population: z.number(),\n rank: z.number(),\n});\n\nconst ResponseSchema = z.object({ cities: z.array(CitySchema) });\n\nconst SYSTEM_PROMPT = `You are a helpful assistant that can extract information from markdown and convert it into a structured format.\nEnsure the output adheres to the following:\n- city: The name of the city\n- country: The name of the country\n- population: The population of the city\n- rank: The rank of the city\n\nProvide the extracted data as a JSON object. Parse the Markdown content carefully to identify and categorize the city details accurately.\n`;\n\nconst main = async () => {\n console.log(\"Started scraping\");\n const scrapeResult = await client.scrape.startAndWait({\n url: \"https://en.wikipedia.org/wiki/List_of_largest_cities\",\n scrapeOptions: {\n // Only return the markdown for the scraped data\n formats: [\"markdown\"],\n // Only include the table element with class `wikitable` from the page\n includeTags: [\".wikitable\"],\n // Remove any img tags from the table\n excludeTags: [\"img\"],\n },\n });\n console.log(\"Finished scraping\");\n if (scrapeResult.status === \"failed\") {\n console.error(\"Scrape failed:\", scrapeResult.error);\n return;\n }\n if (!scrapeResult.data.markdown) {\n console.error(\"No markdown data found in the scrape result\");\n return;\n }\n\n console.log(\"Extracting data from markdown\");\n const completion = await openai.beta.chat.completions.parse({\n model: \"gpt-4o-mini\",\n messages: [\\\n {\\\n role: \"system\",\\\n content: SYSTEM_PROMPT,\\\n },\\\n { role: \"user\", content: scrapeResult.data.markdown },\\\n ],\n response_format: zodResponseFormat(ResponseSchema, \"cities\"),\n });\n console.log(\"Finished extracting data from markdown\");\n\n const cities = completion.choices[0].message.parsed;\n\n const data = JSON.stringify(cities, null, 2);\n fs.writeFileSync(\"cities.json\", data);\n};\n\nmain();\n```\n\nWith just a single call to the SDKs crawl `startAndWait` function, we can get back the exact information we need from the page in properly formatted markdown. To make sure we narrow down the data we get back to just the information we need, we make sure to only include the `wikiTable` class element and remove any unnecessary image tags.\n\nOnce we have the markdown text, we can simply just pass it into the request to the `parse` function from the openai library with the `response_format` we want and we will have our list of the most populous cities.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/extract-information-with-an-llm\\#run-the-scraper) Run the Scraper\n\nOnce you have the code copied, you can run the script with:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnode scraper.js\n```\n\nIf everything completes successfully, you should see a `cities.json` file in your project directory with the data in this format:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"cities\": [\\\n {\\\n \"city\": \"Tokyo\",\\\n \"country\": \"Japan\",\\\n \"population\": 37468000,\\\n \"rank\": 1\\\n },\\\n {\\\n \"city\": \"Delhi\",\\\n \"country\": \"India\",\\\n \"population\": 28514000,\\\n \"rank\": 2\\\n },\\\n {\\\n \"city\": \"Shanghai\",\\\n \"country\": \"China\",\\\n \"population\": 25582000,\\\n \"rank\": 3\\\n },\\\n ...\\\n ]\n}\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/extract-information-with-an-llm\\#next-steps) Next Steps\n\nThis is a simple example, but you can adapt it to scrape more complex data from other sites, or crawl entire websites.\n\n[PreviousAI Function Calling](https://www.docs.hyperbrowser.ai/guides/ai-function-calling) [NextUsing Hyperbrowser Session](https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session)\n\nLast updated 7 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/guides/extract-information-with-an-llm", "title": "Extract Information with an LLM | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/suB5OCCvQWYriQKaOvdC", "og:title": "Extract Information with an LLM | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/guides/extract-information-with-an-llm", "description": "Use Hyperbrowser to scrape a wikipedia page and extract information with an LLM", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/suB5OCCvQWYriQKaOvdC", "twitter:title": "Extract Information with an LLM | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/guides/extract-information-with-an-llm", "og:description": "Use Hyperbrowser to scrape a wikipedia page and extract information with an LLM", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Use Hyperbrowser to scrape a wikipedia page and extract information with an LLM" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/guides/extract-information-with-an-llm", "status": "completed", "error": null }, "summary": "Learn how to extract information using a Large Language Model with Hyperbrowser." }, { "pathname": "/guides/using-hyperbrowser-session", "data": { "markdown": "Using Hyperbrowser Session \\| Hyperbrowser\n\nIn this guide, we will see how to use Hyperbrowser and Puppeteer to create a new session, connect to it, and scrape current weather data.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session\\#setup) Setup\n\nFirst, lets create a new Node.js project.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nmkdir weather-scraper && cd weather-scraper\nnpm init -y\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session\\#installation) Installation\n\nNext, let's install the necessary dependencies to run our script.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk puppeteer-core dotenv\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session\\#setup-your-environment) Setup your Environment\n\nTo use Hyperbrowser with your code, you will need an API Key. You can get one easily from the [dashboard](https://app.hyperbrowser.ai/settings?tab=api_key). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY`.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session\\#code) Code\n\nNext, create a new file `index.js` and add the following code:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\nimport { connect } from \"puppeteer-core\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst main = async () => {\n const location = process.argv[2];\n if (!location) {\n console.error(\"Please provide a location as a command line argument\");\n process.exit(1);\n }\n\n console.log(\"Starting session\");\n const session = await client.sessions.create();\n console.log(\"Session created:\", session.id);\n\n try {\n const browser = await connect({ browserWSEndpoint: session.wsEndpoint });\n\n const [page] = await browser.pages();\n\n await page.goto(\"https://openweathermap.org/city\", {\n waitUntil: \"load\",\n timeout: 20_000,\n });\n await page.waitForSelector(\".search-container\", {\n visible: true,\n timeout: 10_000,\n });\n await page.type(\".search-container input\", location);\n await page.click(\".search button\");\n await page.waitForSelector(\".search-dropdown-menu\", {\n visible: true,\n timeout: 10_000,\n });\n\n const [response] = await Promise.all([\\\n page.waitForNavigation(),\\\n page.click(\".search-dropdown-menu li:first-child\"),\\\n ]);\n\n await page.waitForSelector(\".current-container\", {\n visible: true,\n timeout: 10_000,\n });\n const locationName = await page.$eval(\n \".current-container h2\",\n (el) => el.textContent\n );\n const currentTemp = await page.$eval(\n \".current-container .current-temp\",\n (el) => el.textContent\n );\n const description = await page.$eval(\n \".current-container .bold\",\n (el) => el.textContent\n );\n\n const windInfo = await page.$eval(\".weather-items .wind-line\", (el) =>\n el.textContent.trim()\n );\n const pressureInfo = await page.$eval(\n \".weather-items li:nth-child(2)\",\n (el) => el.textContent.trim()\n );\n const humidityInfo = await page.$eval(\n \".weather-items li:nth-child(3)\",\n (el) => el.textContent.trim()?.split(\":\")[1]\n );\n const dewpoint = await page.$eval(\n \".weather-items li:nth-child(4)\",\n (el) => el.textContent.trim()?.split(\":\")[1]\n );\n const visibility = await page.$eval(\n \".weather-items li:nth-child(5)\",\n (el) => el.textContent.trim()?.split(\":\")[1]\n );\n\n console.log(\"\\nWeather Information:\");\n console.log(\"------------------\");\n console.log(`Location: ${locationName}`);\n console.log(`Temperature: ${currentTemp}`);\n console.log(`Conditions: ${description}`);\n console.log(`Wind: ${windInfo}`);\n console.log(`Pressure: ${pressureInfo}`);\n console.log(`Humidity: ${humidityInfo}`);\n console.log(`Dew Point: ${dewpoint}`);\n console.log(`Visibility: ${visibility}`);\n console.log(\"------------------\\n\");\n\n await page.screenshot({ path: \"screenshot.png\" });\n await page.close();\n await browser.close();\n } catch (error) {\n console.error(`Encountered an error: ${error}`);\n } finally {\n await client.sessions.stop(session.id);\n console.log(\"Session stopped:\", session.id);\n }\n};\n\nmain().catch((error) => {\n console.error(`Encountered an error: ${error}`);\n});\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session\\#run-the-scraper) Run the Scraper\n\nTo run the weather scraper:\n\n1. Open a terminal and navigate to your project directory\n\n2. Run the script with a location argument:\n\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnode index.js \"New York\"\n```\n\nReplace `\"New York\"` with the location you want weather data for.\n\nThe script will:\n\n1. Create a new Hyperbrowser session\n\n2. Launch a Puppeteer browser and connect to the session\n\n3. Navigate to the OpenWeatherMap city page\n\n4. Search for the specified location and hit the Search button\n\n5. Select the first option from a list in a dropdown menu and navigate to that page\n\n6. Scrape the current weather data from the page\n\n7. Print the weather information to the console\n\n8. Save a screenshot of the page\n\n9. Close the browser and stop the Hyperbrowser session\n\n\nYou should see output like:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nWeather Information:\n------------------\nLocation: New York City, US\nTemperature: 9掳C\nConditions: overcast clouds\nWind: Gentle breeze, 3.6 m/s, west-southwest\nPressure: 1013 hPa\nHumidity: 81%\nDew Point: 6掳C\nVisibility: 10 km\n------------------\n```\n\nAnd a `screenshot.png` file saved in your project directory.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session\\#how-it-works) How it Works\n\nLet's break down the key steps:\n\n01. We import the required libraries and load the environment variables\n\n02. We create a new Hyperbrowser client with the API key\n\n03. We start a new Hyperbrowser session with `client.sessions.create()`\n\n04. We launch a Puppeteer browser and connect it to the Hyperbrowser session\n\n05. We navigate to the OpenWeatherMap city page\n\n06. We search for the location provided as a command line argument\n\n07. We wait for the search results and click the first result\n\n08. We scrape the weather data from the page using Puppeteer's `page.$eval` method\n\n09. We print the scraped data, take a screenshot, and save it to disk\n\n10. Finally, we close the browser and stop the Hyperbrowser session\n\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session\\#next-steps) Next Steps\n\nThis example demonstrates a basic weather scraping workflow using a Hyperbrowser session. You can expand on it to:\n\n- Accept multiple locations and fetch weather data for each\n\n- Get the 8-day forecast for the location\n\n- Schedule the script to run periodically and save historical weather data\n\n\n[PreviousExtract Information with an LLM](https://www.docs.hyperbrowser.ai/guides/extract-information-with-an-llm) [NextCAPTCHA Solving](https://www.docs.hyperbrowser.ai/guides/captcha-solving)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/guides/using-hyperbrowser-session", "title": "Using Hyperbrowser Session | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/5x0Pt70fuhhxU5e65DyS", "og:title": "Using Hyperbrowser Session | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session", "description": "Using Hyperbrowser's session", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/5x0Pt70fuhhxU5e65DyS", "twitter:title": "Using Hyperbrowser Session | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/guides/using-hyperbrowser-session", "og:description": "Using Hyperbrowser's session", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Using Hyperbrowser's session" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session", "status": "completed", "error": null }, "summary": "The page describes how to use sessions in Hyperbrowser for efficient web scraping and automation." }, { "pathname": "/guides/captcha-solving", "data": { "markdown": "CAPTCHA Solving \\| Hyperbrowser\n\nHyperbrowser's CAPTCHA solving feature requires being on a `PAID` plan.\n\nIn this guide, we will see how to use Hyperbrowser and its integrated CAPTCHA solver to scrape Today's Top Deals from Amazon without being blocked.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/captcha-solving\\#setup) Setup\n\nFirst, lets create a new Node.js project.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nmkdir amazon-deals-scraper && cd amazon-deals-scraper\nnpm init -y\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/captcha-solving\\#installation) Installation\n\nNext, let's install the necessary dependencies to run our script.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk puppeteer-core dotenv\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/captcha-solving\\#setup-your-environment) Setup your Environment\n\nTo use Hyperbrowser with your code, you will need an API Key. You can get one easily from the [dashboard](https://app.hyperbrowser.ai/settings?tab=api_key). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY`.\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/captcha-solving\\#code) Code\n\nNext, create a new file `index.js` and add the following code:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { config } from \"dotenv\";\nimport { connect } from \"puppeteer-core\";\n\nconfig();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));\n\nconst main = async () => {\n console.log(\"Starting session\");\n const session = await client.sessions.create({\n solveCaptchas: true,\n adblock: true,\n annoyances: true,\n trackers: true,\n });\n console.log(\"Session created:\", session.id);\n\n try {\n const browser = await connect({ browserWSEndpoint: session.wsEndpoint });\n\n const [page] = await browser.pages();\n\n await page.goto(\"https://amazon.com/deals\", {\n waitUntil: \"load\",\n timeout: 20_000,\n });\n\n const pageTitle = await page.title();\n console.log(\"Navigated to Page:\", pageTitle);\n\n await sleep(10_000);\n\n const products = await page.evaluate(() => {\n const items = document.querySelectorAll(\".dcl-carousel-element\");\n return Array.from(items)\n .map((item) => {\n const nameElement = item.querySelector(\".dcl-product-label\");\n const dealPriceElement = item.querySelector(\n \".dcl-product-price-new .a-offscreen\"\n );\n const originalPriceElement = item.querySelector(\n \".dcl-product-price-old .a-offscreen\"\n );\n const percentOffElement = item.querySelector(\n \".dcl-badge .a-size-mini\"\n );\n\n return {\n name: nameElement ? nameElement.textContent.trim() : null,\n dealPrice: dealPriceElement\n ? dealPriceElement.textContent.trim()\n : null,\n originalPrice: originalPriceElement\n ? originalPriceElement.textContent.trim()\n : null,\n percentOff: percentOffElement\n ? percentOffElement.textContent.trim()\n : null,\n };\n })\n .filter((product) => product.name && product.dealPrice);\n });\n\n console.log(\"Found products:\", JSON.stringify(products, null, 2));\n\n await page.close();\n await browser.close();\n } catch (error) {\n console.error(`Encountered an error: ${error}`);\n } finally {\n await client.sessions.stop(session.id);\n console.log(\"Session stopped:\", session.id);\n }\n};\n\nmain().catch((error) => {\n console.error(`Encountered an error: ${error}`);\n});\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/captcha-solving\\#run-the-scraper) Run the Scraper\n\nTo run the Amazon deals scraper:\n\n1. In your terminal, navigate to the project directory\n\n2. Run the script with Node.js:\n\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnode index.js\n```\n\nThe script will:\n\n1. Create a new Hyperbrowser session with captcha solving, ad blocking, and anti-tracking enabled\n\n2. Launch a Puppeteer browser and connect it to the session\n\n3. Navigate to the Amazon deals page, solving any CAPTCHAs that are encountered\n\n4. Wait 10 seconds for the page to load its content\n\n5. Scrape the deal data using Puppeteer's `page.evaluate` method\n\n6. Print the scraped products to the console\n\n7. Close the browser and stop the Hyperbrowser session\n\n\nYou should see the scraped products printed in the console, like:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n[\\\n {\\\n \"name\": \"Apple AirPods Pro\",\\\n \"dealPrice\": \"$197.00\",\\\n \"originalPrice\": \"$249.99\",\\\n \"percentOff\": \"21% off\"\\\n },\\\n {\\\n \"name\": \"Echo Dot (4th Gen)\",\\\n \"dealPrice\": \"$27.99\",\\\n \"originalPrice\": \"$49.99\",\\\n \"percentOff\": \"44% off\"\\\n }\\\n]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/captcha-solving\\#how-it-works) How it Works\n\nLet's break down the key parts:\n\n1. We create a new Hyperbrowser session with `solveCaptchas`, `adblock`, `annoyances`, and `trackers` set to `true`. This enables the captcha solver and other anti-bot evasion features.\n\n2. We launch a Puppeteer browser and connect it to the Hyperbrowser session.\n\n3. We navigate to the Amazon deals page and wait for any CAPTCHAs to be solved automatically by Hyperbrowser.\n\n4. We pause execution for 10 seconds with `sleep` to allow all content to be loaded.\n\n5. We use `page.evaluate` to run JavaScript on the page to scrape the deal data.\n\n6. In the evaluator function, we select the deal elements, extract the relevant data, and return an array of product objects.\n\n7. We print the scraped data, close the browser, and stop the Hyperbrowser session.\n\n\nWithout the `solveCaptchas` enabled, we would encounter a screen like this when trying to navigate to the deals page:\n\n![](https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4095930873-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FfwZVRs9Jmwzw9cfbchYG%252Fuploads%252FUo9d9TCWSxzh65PtYQBx%252Famazon-captcha.png%3Falt%3Dmedia%26token%3D31203321-a09f-4f74-9612-36ae59650a28&width=768&dpr=4&quality=100&sign=6f4ba033&sv=2)\n\nThe captcha solver runs automatically in the background, so we don't need to handle captchas explicitly in our script. If a captcha appears, Hyperbrowser will solve it and continue loading the page. In this case, it would solve this CAPTCHA and continue on to the deals page.\n\n[PreviousUsing Hyperbrowser Session](https://www.docs.hyperbrowser.ai/guides/using-hyperbrowser-session) [NextModel Context Protocol](https://www.docs.hyperbrowser.ai/guides/model-context-protocol)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/guides/captcha-solving", "title": "CAPTCHA Solving | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/Maw8d1ENx9H7ZAiMBu29", "og:title": "CAPTCHA Solving | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/guides/captcha-solving", "description": "Using Hyperbrowser's CAPTCHA Solving", "color-scheme": "dark", "link:preload": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4095930873-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FfwZVRs9Jmwzw9cfbchYG%252Fuploads%252FUo9d9TCWSxzh65PtYQBx%252Famazon-captcha.png%3Falt%3Dmedia%26token%3D31203321-a09f-4f74-9612-36ae59650a28&width=768&dpr=4&quality=100&sign=6f4ba033&sv=2", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/Maw8d1ENx9H7ZAiMBu29", "twitter:title": "CAPTCHA Solving | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/guides/captcha-solving", "og:description": "Using Hyperbrowser's CAPTCHA Solving", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Using Hyperbrowser's CAPTCHA Solving" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/guides/captcha-solving", "status": "completed", "error": null }, "summary": "Hyperbrowser provides tools and guidance for CAPTCHA solving in web scraping and automation processes." }, { "pathname": "/guides/model-context-protocol", "data": { "markdown": "Model Context Protocol \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#overview) Overview\n\nThe MCP server provides a standardized interface for AI models to access Hyperbrowser's web automation capabilities. This server implementation supports key functions like web scraping, structured data extraction, and web crawling.\n\nYou can see the MCP server code at [https://github.com/hyperbrowserai/mcp](https://github.com/hyperbrowserai/mcp)\n\n![](https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4095930873-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FfwZVRs9Jmwzw9cfbchYG%252Fuploads%252Fs8PH64mIx6TMQxQQyz2Q%252FScreenshot%25202025-03-12%2520at%252014.30.07.png%3Falt%3Dmedia%26token%3D6ff6268d-924b-47f0-98ea-e17afde47554&width=768&dpr=4&quality=100&sign=a00ecfb&sv=2)\n\nWith the hyperbrowser MCP, claude can browse the web!\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#installation) Installation\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#prerequisites) Prerequisites\n\n- Node.js (v14 or later)\n\n- npm or yarn package manager\n\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#setup) Setup\n\n1. Install the MCP server for Hyperbrowser\n\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpx hyperbrowser-mcp\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#configuration) Configuration\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#client-setup) Client Setup\n\nConfigure your MCP client to connect to the Hyperbrowser MCP server:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"mcpServers\": {\n \"hyperbrowser\": {\n \"command\": \"npx\",\n \"args\": [\"hyperbrowser-mcp\"],\n \"env\": {\n \"HYPERBROWSER_API_KEY\": \"your-api-key\"\n }\n }\n }\n}\n```\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#alternative-setup-using-shell-script) Alternative Setup Using Shell Script\n\nFor clients that don't support the `env` field (like Cursor):\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"mcpServers\": {\n \"hyperbrowser\": {\n \"command\": \"bash\",\n \"args\": [\"/path/to/hyperbrowser-mcp/run_server.sh\"]\n }\n }\n}\n```\n\nEdit `run_server.sh` to include your API key:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n#!/bin/bash\nexport HB_API_KEY=\"your-api-key\"\nnpx hyperbrowser-mcp\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#tools) Tools\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#scrape-webpage) Scrape Webpage\n\nRetrieves content from a specified URL in various formats.\n\nMethod: `scrape_webpage`\n\nParameters:\n\n- `url`: string - The URL to scrape\n\n- `outputFormat`: string\\[\\] - Desired output formats ( `markdown`, `html`, `links`, `screenshot`)\n\n- `apiKey`: string (optional) - API key for authentication\n\n- `sessionOptions`: object (optional) - Browser session configuration\n\n\nExample:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"url\": \"https://example.com\",\n \"outputFormat\": [\"markdown\", \"screenshot\"],\n \"sessionOptions\": {\n \"useStealth\": true,\n \"acceptCookies\": true\n }\n}\n```\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#extract-structured-data) Extract Structured Data\n\nExtracts data from webpages according to a specified schema.\n\nMethod: `extract_structured_data`\n\nParameters:\n\n- `urls`: string\\[\\] - List of URLs to extract data from (supports wildcards)\n\n- `prompt`: string - Instructions for extraction\n\n- `schema`: object (optional) - JSON schema for the extracted data\n\n- `apiKey`: string (optional) - API key for authentication\n\n- `sessionOptions`: object (optional) - Browser session configuration\n\n\nExample:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"urls\": [\"https://example.com/products/*\"],\n \"prompt\": \"Extract product name, price, and description\",\n \"schema\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": { \"type\": \"string\" },\n \"price\": { \"type\": \"number\" },\n \"description\": { \"type\": \"string\" }\n }\n },\n \"sessionOptions\": {\n \"useStealth\": true\n }\n}\n```\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#crawl-webpages) Crawl Webpages\n\nNavigates through multiple pages on a website, optionally following links.\n\nMethod: `crawl_webpages`\n\nParameters:\n\n- `url`: string - Starting URL for crawling\n\n- `outputFormat`: string\\[\\] - Desired output formats\n\n- `followLinks`: boolean - Whether to follow page links\n\n- `maxPages`: number (default: 10) - Maximum pages to crawl\n\n- `ignoreSitemap`: boolean (optional) - Skip using site's sitemap\n\n- `apiKey`: string (optional) - API key for authentication\n\n- `sessionOptions`: object (optional) - Browser session configuration\n\n\nExample:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"url\": \"https://example.com\",\n \"outputFormat\": [\"markdown\", \"links\"],\n \"followLinks\": true,\n \"maxPages\": 5,\n \"sessionOptions\": {\n \"acceptCookies\": true\n }\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/guides/model-context-protocol\\#session-options) Session Options\n\nAll tools support these common session configuration options:\n\n- `useStealth`: boolean - Makes browser detection more difficult\n\n- `useProxy`: boolean - Routes traffic through proxy servers\n\n- `solveCaptchas`: boolean - Automatically solves CAPTCHA challenges\n\n- `acceptCookies`: boolean - Automatically handles cookie consent popups\n\n\n[PreviousCAPTCHA Solving](https://www.docs.hyperbrowser.ai/guides/captcha-solving) [NextSDKs](https://www.docs.hyperbrowser.ai/reference/sdks)\n\nLast updated 7 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/guides/model-context-protocol", "title": "Model Context Protocol | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/WqfvWymEio29TZfvaACB", "og:title": "Model Context Protocol | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/guides/model-context-protocol", "description": "Using the MCP server for Hyperbrowser integration.", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/WqfvWymEio29TZfvaACB", "twitter:title": "Model Context Protocol | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/guides/model-context-protocol", "og:description": "Using the MCP server for Hyperbrowser integration.", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Using the MCP server for Hyperbrowser integration." }, "html": null, "url": "https://www.docs.hyperbrowser.ai/guides/model-context-protocol", "status": "completed", "error": null }, "summary": "The page covers the Model Context Protocol used in Hyperbrowser for web scraping and automation." }, { "pathname": "/reference/sdks", "data": { "markdown": "SDKs \\| Hyperbrowser\n\nHyperbrowser provides SDKs in Node and Python.\n\n[PreviousModel Context Protocol](https://www.docs.hyperbrowser.ai/guides/model-context-protocol) [NextNode](https://www.docs.hyperbrowser.ai/reference/sdks/node)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks", "title": "SDKs | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/76jhRUJsCzNtlmGnxfoK", "og:title": "SDKs | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/76jhRUJsCzNtlmGnxfoK", "twitter:title": "SDKs | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks", "status": "completed", "error": null }, "summary": "The page discusses Hyperbrowser SDKs for web scraping and automation, including features and usage details." }, { "pathname": "/reference/sdks/node", "data": { "markdown": "Node \\| Hyperbrowser\n\nView on [Github](https://github.com/hyperbrowserai/node-sdk)\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node\\#installation) Installation\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nnpm install @hyperbrowser/sdk\n```\n\nor\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nyarn add @hyperbrowser/sdk\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node\\#usage) Usage\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { connect } from \"puppeteer-core\";\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport dotenv from \"dotenv\";\n\ndotenv.config();\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\n(async () => {\n const session = await client.sessions.create();\n\n const browser = await connect({\n browserWSEndpoint: session.wsEndpoint,\n defaultViewport: null,\n });\n\n // Create a new page\n const [page] = await browser.pages();\n\n // Navigate to a website\n console.log(\"Navigating to Hacker News...\");\n await page.goto(\"https://news.ycombinator.com/\");\n const pageTitle = await page.title();\n console.log(\"Page title:\", pageTitle);\n\n await page.close();\n await browser.close();\n console.log(\"Session completed!\");\n await client.sessions.stop(session.id);\n})().catch((error) => console.error(error.message));\n```\n\n[PreviousSDKs](https://www.docs.hyperbrowser.ai/reference/sdks) [NextSessions](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/node", "title": "Node | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/a9Qy6LKDkuZjQpkS2oNT", "og:title": "Node | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/node", "description": "Learn about Hyperbrowser's Node SDK", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/a9Qy6LKDkuZjQpkS2oNT", "twitter:title": "Node | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/node", "og:description": "Learn about Hyperbrowser's Node SDK", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Learn about Hyperbrowser's Node SDK" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/node", "status": "completed", "error": null }, "summary": "Overview of the Node module in Hyperbrowser for web scraping and automation." }, { "pathname": "/reference/sdks/node/sessions", "data": { "markdown": "Sessions \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#create-session) Create Session\n\nCreates a new browser session with optional configuration.\n\n**Method:** `client.sessions.create(params?: CreateSessionParams): Promise`\n\n**Endpoint:** `POST /api/session`\n\n**Parameters:**\n\n- `CreateSessionParams`:\n\n\n\n- `useStealth?: boolean` \\- Use stealth mode.\n\n- `useProxy?: boolean` \\- Use proxy.\n\n- `proxyServer?: string` \\- Proxy server URL to route the session through.\n\n- `proxyServerUsername?: string` \\- Username for proxy server authentication.\n\n- `proxyServerPassword?: string` \\- Password for proxy server authentication.\n\n- `proxyCountry?:` [`Country`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#country) \\- Desired proxy country.\n\n- proxyState?: [`State`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#state)\\- Desired State. Is mutually exclusive with proxyCity. Currently only US states are supported. States need to be in two letter codes\n\n- proxyCity?: string - Desired City. Is mutually exclusive with proxyState. Some cities might not be supported, so before using a new city, we recommend trying it out.\n\n- `operatingSystems?: ` [`OperatingSystem`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#operatingsystem) `[]` \\- Preferred operating systems for the session. Possible values are:\n\n\n\n- `OperatingSystem.WINDOWS`\n\n- `OperatingSystem.ANDROID`\n\n- `OperatingSystem.MACOS`\n\n- `OperatingSystem.LINUX`\n\n- `OperatingSystem.IOS`\n\n\n- `device?: (\"desktop\" | \"mobile\")[]` \\- Preferred device types. Possible values are:\n\n\n\n- `\"desktop\"`\n\n- `\"mobile\"`\n\n\n- `platform?: ` [`Platform`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#platform) `[]` \\- Preferred browser platforms. Possible values are:\n\n\n\n- `Platform.CHROME`\n\n- `Platform.FIREFOX`\n\n- `Platform.SAFARI`\n\n- `Platform.EDGE`\n\n\n- `locales?: ` [`ISO639_1`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#iso639_1) `[]` \\- Preferred locales (languages) for the session. Use ISO 639-1 codes.\n\n- `screen?: ` [`ScreenConfig`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#screenconfig) \\- Screen configuration for the session.\n\n\n\n- `width: number` \\- Screen width.\n\n- `height: number` \\- Screen height.\n\n\n- `solveCaptchas?: boolean` \\- Solve captchas.\n\n- `adblock?: boolean` \\- Block ads.\n\n- `trackers?: boolean` \\- Block trackers.\n\n- `annoyances?: boolean` \\- Block annoyances.\n\n- `enableWebRecording?: boolean` \\- Default true\n\n- `extensionIds?: string[]` \\- Array of extension Ids\n\n- `acceptCookies?: boolean` \\- Automatically Accept Cookies on the page\n\n- `urlBlocklist?: string[]`\n\n- `browserArgs?: string[]`\n\n\n**Response:** [`SessionDetail`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#sessiondetail)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst session = await client.sessions.create();\nconsole.log(session.id);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#get-session-details) Get Session Details\n\nRetrieves details of a specific session.\n\n**Method:** `client.sessions.get(id: string): Promise`\n\n**Endpoint:** `GET /api/session/{id}`\n\n**Parameters:**\n\n- `id: string` \\- Session ID\n\n\n**Response:** [`SessionDetail`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#sessiondetail)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst session = await client.sessions.get(\"182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e\");\nconsole.log(session.id);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#list-sessions) [鈥媇(https://docs.hyperbrowser.ai/sdks/node\\#list-sessions) List Sessions\n\nRetrieves a list of all sessions with optional filtering.\n\n**Method:** `client.sessions.list(params?: SessionListParams): Promise`\n\n**Endpoint:** `GET /api/sessions`\n\n**Parameters:**\n\n- `SessionListParams`:\n\n\n\n- `status?: \"active\" | \"closed\" | \"error\"` \\- Filter sessions by status\n\n- `page?: number` \\- Page number for pagination\n\n\n**Response:** [`SessionListResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#sessionlistresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.sessions.list({\n status: \"active\",\n page: 1,\n});\nconsole.log(response.sessions);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#stop-session) [鈥媇(https://docs.hyperbrowser.ai/sdks/node\\#stop-session) Stop Session\n\nStops a running session.\n\n**Method:** `client.sessions.stop(id: string): Promise`\n\n**Endpoint:** `PUT /api/session/{id}/stop`\n\n**Parameters:**\n\n- `id: string` \\- Session ID\n\n\n**Response:** [`BasicResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#basicresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.sessions.stop(\n \"182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e\"\n);\nconsole.log(`Session stopped: ${response.success}`);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#get-session-recording) Get Session Recording\n\nGet the recording of a session.\n\n**Method:** `client.sessions.getRecording(id: string): Promise`\n\n**Endpoint:** `GET /api/session/{id}/recording`\n\n**Parameters:**\n\n- `id: string` \\- Session ID\n\n\n**Response:** [`SessionRecording`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#sessionrecording) `[]`\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst recordingData = await client.sessions.getRecording(\n \"182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e\"\n);\nconsole.log(recordingData);\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#types) Types\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#sessionstatus) SessionStatus\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype SessionStatus = \"active\" | \"closed\" | \"error\";\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#country) Country\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype Country =\n | \"AD\"\n | \"AE\"\n | \"AF\"\n ...\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#state) State\n\nCurrently only US States are supported. Standard two letter state codes are used.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype State =\n | \"AL\"\n | \"AK\"\n | \"AZ\"\n ...\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#operatingsystem) OperatingSystem\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype OperatingSystem = \"windows\" | \"android\" | \"macos\" | \"linux\" | \"ios\";\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#platform) Platform\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype Platform = \"chrome\" | \"firefox\" | \"safari\" | \"edge\";\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#iso639_1) ISO639\\_1\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype ISO639_1 =\n | \"aa\"\n | \"ab\"\n | \"ae\"\n ...\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#basicresponse) BasicResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface BasicResponse {\n success: boolean;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#session) Session\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface Session {\n id: string;\n teamId: string;\n status: SessionStatus;\n startTime?: number;\n endTime?: number;\n createdAt: string;\n updatedAt: string;\n sessionUrl: string;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#sessiondetail) SessionDetail\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface SessionDetail extends Session {\n wsEndpoint?: string;\n liveUrl?: string;\n token?: string;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#sessionlistresponse) SessionListResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface SessionListResponse {\n sessions: Session[];\n totalCount: number;\n page: number;\n perPage: number;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#screenconfig) ScreenConfig\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface ScreenConfig {\n width: number;\n height: number;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#createsessionparams) CreateSessionParams\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface CreateSessionParams {\n useStealth?: boolean;\n useProxy?: boolean;\n proxyServer?: string;\n proxyServerPassword?: string;\n proxyServerUsername?: string;\n proxyCountry?: Country;\n operatingSystems?: OperatingSystem[];\n device?: (\"desktop\" | \"mobile\")[];\n platform?: Platform[];\n locales?: ISO639_1[];\n screen?: ScreenConfig;\n solveCaptchas?: boolean;\n adblock?: boolean;\n trackers?: boolean;\n annoyances?: boolean;\n enableWebRecording?: boolean;\n extensionIds?: string[];\n acceptCookies?: boolean;\n urlBlocklist?: string[];\n browserArgs?: string[];\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions\\#sessionrecording) SessionRecording\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface SessionRecording {\n type: number;\n data: unknown;\n timestamp: number;\n delay?: number;\n}\n```\n\n[PreviousNode](https://www.docs.hyperbrowser.ai/reference/sdks/node) [NextProfiles](https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles)\n\nLast updated 22 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/node/sessions", "title": "Sessions | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/zmW3nPwbhAePdtm3HgyZ", "og:title": "Sessions | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/zmW3nPwbhAePdtm3HgyZ", "twitter:title": "Sessions | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/node/sessions", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions", "status": "completed", "error": null }, "summary": "The page discusses sessions in Hyperbrowser, detailing how to manage and utilize them effectively for web scraping tasks." }, { "pathname": "/reference/sdks/node/profiles", "data": { "markdown": "Profiles \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles\\#create-session) Create Profile\n\nCreates a new profile.\n\n**Method:** `client.profiles.create(): Promise`\n\n**Endpoint:** `POST /api/profile`\n\n**Response:** [`CreateProfileResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles#createprofileresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst profile = await client.profiles.create();\nconsole.log(profile.id);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles\\#create-session-1) Get Profile\n\nGet details of an existing profile.\n\n**Method:** `client.profiles.get(id: string): Promise`\n\n**Endpoint:** `GET /api/profile/{id}`\n\n**Parameters:**\n\n- `id: string` \\- Profile ID\n\n\n**Response:** [`ProfileResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles#profileresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst profile = await client.profiles.get(\"36946080-9b81-4288-81d5-b15f2191f222\");\nconsole.log(profile.id);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles\\#create-session-2) Delete Profile\n\nDelete an existing profile.\n\n**Method:** `client.profiles.delete(id: string): Promise`\n\n**Endpoint:** `DELETE /api/profile/{id}`\n\n**Parameters:**\n\n- `id: string` \\- Profile ID\n\n\n**Response:** [`BasicResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#basicresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.profiles.delete(\"36946080-9b81-4288-81d5-b15f2191f222\");\nconsole.log(response);\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles\\#types) Types\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles\\#createprofileresponse) CreateProfileResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface CreateProfileResponse {\n id: string;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles\\#profileresponse) ProfileResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface ProfileResponse {\n id: string;\n teamId: string;\n createdAt: string;\n updatedAt: string;\n}\n```\n\n[PreviousSessions](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions) [NextScrape](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/node/profiles", "title": "Profiles | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/KL7L1tX0A1fKLEVNG0Vi", "og:title": "Profiles | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/KL7L1tX0A1fKLEVNG0Vi", "twitter:title": "Profiles | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/node/profiles", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles", "status": "completed", "error": null }, "summary": "The \"Profiles\" page of Hyperbrowser covers user profiles and their management within the tool." }, { "pathname": "/reference/sdks/node/scrape", "data": { "markdown": "Scrape \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape\\#start-scrape-job) Start Scrape Job\n\nStarts a scrape job for a given URL.\n\n**Method:** `client.scrape.start(params: StartScrapeJobParams): Promise`\n\n**Endpoint:** `POST /api/scrape`\n\n**Parameters:**\n\n- `StartScrapeJobParams`:\n\n\n\n- `url: string` \\- URL to scrape\n\n- `sessionOptions?: ` [`CreateSessionParams`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#createsessionparams)\n\n- `scrapeOptions?: ` [`ScrapeOptions`](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape#scrapeoptions)\n\n\n**Response:** [`StartScrapeJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape#startscrapejobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.scrape.start({\n url: \"https://example.com\",\n});\nconsole.log(response.jobId);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape\\#get-scrape-job) [鈥媇(https://docs.hyperbrowser.ai/sdks/node\\#get-scrape-job) Get Scrape Job\n\nRetrieves details of a specific scrape job.\n\n**Method:** `client.scrape.get(id: string): Promise`\n\n**Endpoint:** `GET /api/scrape/{id}`\n\n**Parameters:**\n\n- `id: string` \\- Scrape job ID\n\n\n**Response:** [`ScrapeJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape#scrapejobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.scrape.get(\n \"182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e\"\n);\nconsole.log(response.status);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape\\#start-scrape-job-and-wait) Start Scrape Job and Wait\n\nStart a scrape job and wait for it to complete\n\n**Method**: `client.scrape.startAndWait(params: StartScrapeJobParams): Promise`\n\n**Parameters:**\n\n- `StartScrapeJobParams`:\n\n\n\n- `url: string` \\- URL to scrape\n\n- `sessionOptions?: ` [`CreateSessionParams`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#createsessionparams)\n\n- `scrapeOptions?: ` [`ScrapeOptions`](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape#scrapeoptions)\n\n\n**Response:** [`ScrapeJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape#scrapejobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.scrape.startAndWait({\n url: \"https://example.com\"\n});\nconsole.log(response.status);\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape\\#types) Types\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape\\#scrapeformat) ScrapeFormat\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype ScrapeFormat = \"markdown\" | \"html\" | \"links\" | \"screenshot\";\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape\\#scrapejobstatus) ScrapeJobStatus\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype ScrapeJobStatus = \"pending\" | \"running\" | \"completed\" | \"failed\";\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape\\#scrapeoptions) ScrapeOptions\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface ScrapeOptions {\n formats?: ScrapeFormat[];\n includeTags?: string[];\n excludeTags?: string[];\n onlyMainContent?: boolean;\n waitFor?: number;\n timeout?: number;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape\\#startscrapejobresponse) StartScrapeJobResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface StartScrapeJobResponse {\n jobId: string;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape\\#scrapejobdata) ScrapeJobData\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface ScrapeJobData {\n metadata?: Record;\n markdown?: string;\n html?: string;\n links?: string[];\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape\\#scrapejobresponse) ScrapeJobResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface ScrapeJobResponse {\n jobId: string;\n status: ScrapeJobStatus;\n data?: ScrapeJobData;\n error?: string;\n}\n```\n\n[PreviousProfiles](https://www.docs.hyperbrowser.ai/reference/sdks/node/profiles) [NextCrawl](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/node/scrape", "title": "Scrape | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/o8yAG3lStiUcsEObBzbO", "og:title": "Scrape | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/o8yAG3lStiUcsEObBzbO", "twitter:title": "Scrape | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/node/scrape", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape", "status": "completed", "error": null }, "summary": "The \"Scrape\" page of Hyperbrowser outlines techniques and tools for web scraping and automation." }, { "pathname": "/reference/sdks/node/crawl", "data": { "markdown": "Crawl \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl\\#start-scrape-job) Start Crawl Job\n\nStarts a crawl job for a given URL.\n\n**Method:** `client.crawl.start(params: StartCrawlJobParams): Promise`\n\n**Endpoint:** `POST /api/crawl`\n\n**Parameters:**\n\n- `StartCrawlJobParams`:\n\n\n\n- `url: string` \\- URL to scrape\n\n- `maxPages?: number` \\- Max number of pages to crawl\n\n- `followLinks?: boolean` \\- Follow links on the page\n\n- `ignoreSitemap?: boolean` \\- Ignore sitemap when finding links to crawl\n\n- `excludePatterns?: string[]` \\- Patterns for paths to exclude from crawl\n\n- `includePatterns?: string[]` \\- Patterns for paths to include in the crawl\n\n- `sessionOptions?: ` [`CreateSessionParams`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#createsessionparams)\n\n- `scrapeOptions?: ` [`ScrapeOptions`](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape#scrapeoptions)\n\n\n**Response:** [`StartCrawlJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl#startcrawljobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.crawl.start({\n url: \"https://example.com\",\n});\nconsole.log(response.jobId);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl\\#get-scrape-job) [鈥媇(https://docs.hyperbrowser.ai/sdks/node\\#get-scrape-job) Get Crawl Job\n\nRetrieves details of a specific crawl job.\n\n**Method:** `client.crawl.get(id: string): Promise`\n\n**Endpoint:** `GET /api/crawl/{id}`\n\n**Parameters:**\n\n- `id: string` \\- Crawl job ID\n\n\n**Response:** [`CrawlJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl#crawljobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.crawl.get(\n \"182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e\"\n);\nconsole.log(response.status);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl\\#start-crawl-job-and-wait) Start Crawl Job and Wait\n\nStart a crawl job and wait for it to complete\n\n**Method**: `client.crawl.startAndWait(params: StartCrawlJobParams, returnAllPages: boolean = true): Promise`\n\n**Parameters:**\n\n- `StartCrawlJobParams`:\n\n\n\n- `url: string` \\- URL to scrape\n\n- `maxPages?: number` \\- Max number of pages to crawl\n\n- `followLinks?: boolean` \\- Follow links on the page\n\n- `ignoreSitemap?: boolean` \\- Ignore sitemap when finding links to crawl\n\n- `excludePatterns?: string[]` \\- Patterns for paths to exclude from crawl\n\n- `includePatterns?: string[]` \\- Patterns for paths to include in the crawl\n\n- `sessionOptions?: ` [`CreateSessionParams`](https://www.docs.hyperbrowser.ai/reference/sdks/node/sessions#createsessionparams)\n\n- `scrapeOptions?: ` [`ScrapeOptions`](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape#scrapeoptions)\n\n\n- `returnAllPages: boolean` \\- Return all pages in the crawl job response\n\n\n**Response:** [`CrawlJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl#crawljobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.crawl.startAndWait({\n url: \"https://example.com\"\n});\nconsole.log(response.status);\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl\\#types) Types\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl\\#crawlpagestatus) CrawlPageStatus\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype CrawlPageStatus = \"completed\" | \"failed\";\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl\\#crawljobstatus) CrawlJobStatus\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype CrawlJobStatus = \"pending\" | \"running\" | \"completed\" | \"failed\";\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl\\#startcrawljobresponse) StartCrawlJobResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface StartCrawlJobResponse {\n jobId: string;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl\\#crawledpage) CrawledPage\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface CrawledPage {\n url: string;\n status: CrawlPageStatus;\n error?: string | null;\n metadata?: Record;\n markdown?: string;\n html?: string;\n links?: string[];\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl\\#crawljobresponse) CrawlJobResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface CrawlJobResponse {\n jobId: string;\n status: CrawlJobStatus;\n data?: CrawledPage[];\n error?: string;\n totalCrawledPages: number;\n totalPageBatches: number;\n currentPageBatch: number;\n batchSize: number;\n}\n```\n\n[PreviousScrape](https://www.docs.hyperbrowser.ai/reference/sdks/node/scrape) [NextExtensions](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/node/crawl", "title": "Crawl | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/OeeJpUO9uZOYcCgpI4iF", "og:title": "Crawl | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/OeeJpUO9uZOYcCgpI4iF", "twitter:title": "Crawl | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/node/crawl", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl", "status": "completed", "error": null }, "summary": "The \"Crawl\" page of Hyperbrowser details how to use the tool for web scraping and automated data extraction." }, { "pathname": "/reference/sdks/node/extensions", "data": { "markdown": "Extensions \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions\\#add-an-extension) Add an extension\n\nAdds a new chrome **manifest V3** extension\n\n**Method:** `client.extensions.create(params:` CreateExtensionParams `): Promise<` ExtensionResponse `>`\n\n**Endpoint:** `POST /api/extensions/add`\n\n**Parameters:**\n\n- params: [CreateExtensionParams](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions#createextensionparams)\n\n\n\n- filePath: string - Path to the zip containing the manifest V3 compliant extension\n\n- name?: string - Optional name to give to the extension. Does not affect functionality.\n\n\n**Response:** [**ExtensionResponse**](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions#extensionresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst response = await client.extensions.create({\n filePath: \"/Users/test-user/Downloads/extension.zip\",\n});\nconsole.log(response);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions\\#list-all-available-extension) List all available extension\n\nList all available extensions\n\n**Method:** `client.extensions.list(` `): Promise<` ` ListExtensionsResponse >`\n\n**Endpoint:** `POST /api/extensions/list`\n\n**Parameters:**\n\n- N/A\n\n\n**Response:** [**ListExtensionResponse**](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions#listextensionsresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nconst extensionsList = await client.extensions.list();\nfor (let i=0;i `, JSON.stringify(extension))\n}\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions\\#types) Types\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions\\#createextensionparams) CreateExtensionParams\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface CreateExtensionParams {\n filePath: string;\n name?: string;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions\\#extensionresponse) **ExtensionResponse**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ninterface ExtensionResponse {\n name: string;\n id: string;\n createdAt: string;\n updatedAt: string;\n}\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions\\#listextensionsresponse) ListExtensionsResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ntype ListExtensionsResponse = Array\n```\n\n[PreviousCrawl](https://www.docs.hyperbrowser.ai/reference/sdks/node/crawl) [NextPython](https://www.docs.hyperbrowser.ai/reference/sdks/python)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/node/extensions", "title": "Extensions | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/AOYaVpp5NvdyvXEvZOpp", "og:title": "Extensions | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/AOYaVpp5NvdyvXEvZOpp", "twitter:title": "Extensions | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/node/extensions", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions", "status": "completed", "error": null }, "summary": "The Extensions page for Hyperbrowser details available extensions that enhance web scraping and automation functionalities." }, { "pathname": "/reference/sdks/python", "data": { "markdown": "Python \\| Hyperbrowser\n\nView on [Github](https://github.com/hyperbrowserai/python-sdk)\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python\\#installation) Installation\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install hyperbrowser\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python\\#usage) Usage\n\nHyperbrowser's Python SDK includes both a sync and async client.\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python\\#async) Async\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nimport asyncio\nfrom pyppeteer import connect\nfrom hyperbrowser import AsyncHyperbrowser\n\nload_dotenv()\n\nclient = AsyncHyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\nasync def main():\n session = await client.sessions.create()\n\n ws_endpoint = session.ws_endpoint\n browser = await connect(browserWSEndpoint=ws_endpoint, defaultViewport=None)\n\n # Get pages\n pages = await browser.pages()\n if not pages:\n raise Exception(\"No pages available\")\n\n page = pages[0]\n\n # Navigate to a website\n print(\"Navigating to Hacker News...\")\n await page.goto(\"https://news.ycombinator.com/\")\n page_title = await page.title()\n print(\"Page title:\", page_title)\n\n await page.close()\n await browser.disconnect()\n await client.sessions.stop(session.id)\n print(\"Session completed!\")\n\n# Run the asyncio event loop\nasyncio.get_event_loop().run_until_complete(main())\n```\n\n#### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python\\#sync) Sync\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport os\nfrom dotenv import load_dotenv\nfrom playwright.sync_api import sync_playwright\nfrom hyperbrowser import Hyperbrowser\n\nload_dotenv()\n\nclient = Hyperbrowser(api_key=os.getenv(\"HYPERBROWSER_API_KEY\"))\n\ndef main():\n session = client.sessions.create()\n\n ws_endpoint = session.ws_endpoint\n\n # Launch Playwright and connect to the remote browser\n with sync_playwright() as p:\n browser = p.chromium.connect_over_cdp(ws_endpoint)\n context = browser.new_context()\n\n # Get the first page or create a new one\n if len(context.pages) == 0:\n page = context.new_page()\n else:\n page = context.pages[0]\n\n # Navigate to a website\n print(\"Navigating to Hacker News...\")\n page.goto(\"https://news.ycombinator.com/\")\n page_title = page.title()\n print(\"Page title:\", page_title)\n\n page.close()\n browser.close()\n print(\"Session completed!\")\n client.sessions.stop(session.id)\n\nmain()\n```\n\n[PreviousExtensions](https://www.docs.hyperbrowser.ai/reference/sdks/node/extensions) [NextSessions](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/python", "title": "Python | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/UlfIuu4Cfcyz4NTtJrOK", "og:title": "Python | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/python", "description": "Learn about Hyperbrowser's Python SDK", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/UlfIuu4Cfcyz4NTtJrOK", "twitter:title": "Python | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/python", "og:description": "Learn about Hyperbrowser's Python SDK", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Learn about Hyperbrowser's Python SDK" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/python", "status": "completed", "error": null }, "summary": "The page provides documentation on using Hyperbrowser with Python for web scraping and automation tasks." }, { "pathname": "/reference/sdks/python/sessions", "data": { "markdown": "Sessions \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#create-session) Create Session\n\nCreates a new browser session with optional configuration.\n\n**Method:** `client.sessions.create(params?: CreateSessionParams): SessionDetail`\n\n**Endpoint:** `POST /api/session`\n\n**Parameters:**\n\n- `CreateSessionParams`:\n\n\n\n- `use_stealth?: boolean` \\- Use stealth mode.\n\n- `use_proxy?: boolean` \\- Use proxy.\n\n- `proxy_server?: string` \\- Proxy server URL to route the session through.\n\n- `proxy_server_username?: string` \\- Username for proxy server authentication.\n\n- `proxy_server_password?: string` \\- Password for proxy server authentication.\n\n- `proxy_country?: ` [`Country`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#country) \\- Desired proxy country.\n\n- proxy\\_state?: [`State`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#states) \\- Desired State. Is mutually exclusive with proxy\\_city. Currently only US states are supported. States need to be in two letter codes\n\n- proxy\\_city?: string - Desired City. Is mutually exclusive with proxy\\_state. Some cities might not be supported, so before using a new city, we recommend trying it out.\n\n- `operating_systems?: ` [`OperatingSystem`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#operatingsystem) `[]` \\- Preferred operating systems for the session. Possible values are:\n\n\n\n- `OperatingSystem.WINDOWS`\n\n- `OperatingSystem.ANDROID`\n\n- `OperatingSystem.MACOS`\n\n- `OperatingSystem.LINUX`\n\n- `OperatingSystem.IOS`\n\n\n- `device?: (\"desktop\" | \"mobile\")[]` \\- Preferred device types. Possible values are:\n\n\n\n- `\"desktop\"`\n\n- `\"mobile\"`\n\n\n- `platform?: ` [`Platform`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#platform) `[]` \\- Preferred browser platforms. Possible values are:\n\n\n\n- `Platform.CHROME`\n\n- `Platform.FIREFOX`\n\n- `Platform.SAFARI`\n\n- `Platform.EDGE`\n\n\n- `locales?: ` [`ISO639_1`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#iso639_1) `[]` \\- Preferred locales (languages) for the session. Use ISO 639-1 codes.\n\n- `screen?: ` [`ScreenConfig`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#screenconfig) \\- Screen configuration for the session.\n\n\n\n- `width: number` \\- Screen width.\n\n- `height: number` \\- Screen height.\n\n\n- `solve_captchas?: boolean` \\- Solve captchas.\n\n- `adblock?: boolean` \\- Block ads.\n\n- `trackers?: boolean` \\- Block trackers.\n\n- `annoyances?: boolean` \\- Block annoyances.\n\n- `enable_web_recording?: boolean` \\- Default true\n\n- `extension_ids?: string[]` \\- Array of extension Ids\n\n- `accept_cookies?: boolean` \\- Automatically Accept Cookies on the page\n\n- `url_blocklist?: string[]`\n\n- `browser_args?: string[]`\n\n\n**Response:** [`SessionDetail`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#sessiondetail)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nsession = client.sessions.create()\nprint(session.id)\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#get-session-details) Get Session Details\n\nRetrieves details of a specific session.\n\n**Method:** `client.sessions.get(id: str): SessionDetail`\n\n**Endpoint:** `GET /api/session/{id}`\n\n**Parameters:**\n\n- `id: string` \\- Session ID\n\n\n**Response:** [`SessionDetail`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#sessiondetail)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nsession = client.sessions.get(\"182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e\")\nprint(session.id)\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#list-sessions) [鈥媇(https://docs.hyperbrowser.ai/sdks/node\\#list-sessions) List Sessions\n\nRetrieves a list of all sessions with optional filtering.\n\n**Method:** `client.sessions.list(params?: SessionListParams): SessionListResponse`\n\n**Endpoint:** `GET /api/sessions`\n\n**Parameters:**\n\n- `SessionListParams`:\n\n\n\n- `status?: \"active\" | \"closed\" | \"error\"` \\- Filter sessions by status\n\n- `page?: number` \\- Page number for pagination\n\n\n**Response:** [`SessionListResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#sessionlistresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nresponse = client.sessions.list()\nprint(response.sessions)\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#stop-session) [鈥媇(https://docs.hyperbrowser.ai/sdks/node\\#stop-session) Stop Session\n\nStops a running session.\n\n**Method:** `client.sessions.stop(id: str): BasicResponse`\n\n**Endpoint:** `PUT /api/session/{id}/stop`\n\n**Parameters:**\n\n- `id: string` \\- Session ID\n\n\n**Response:** [`BasicResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#basicresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nresponse = client.sessions.stop(\n \"182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e\"\n)\nprint(f\"Session stopped: {response.success}\")\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#get-session-recording) Get Session Recording\n\nGet the recording of a session.\n\n**Method:** `client.sessions.get_recording(id: str): SessionRecording[]`\n\n**Endpoint:** `GET /api/session/{id}/recording`\n\n**Parameters:**\n\n- `id: string` \\- Session ID\n\n\n**Response:** [`SessionRecording`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#sessionrecording) `[]`\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nrecordingData = client.sessions.get_recording(\n \"182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e\"\n);\nprint(recordingData)\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#types) Types\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#sessionstatus) SessionStatus\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nSessionStatus = Literal[\"active\", \"closed\", \"error\"]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#country) Country\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nCountry = Literal[\"AD\", \"AE\", \"AF\", ...]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#states) States\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nState = Literal[\"AL\", \"AK\", \"AZ\", ...]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#operatingsystem) OperatingSystem\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nOperatingSystem = Literal[\"windows\", \"android\", \"macos\", \"linux\", \"ios\"]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#platform) Platform\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nPlatform = Literal[\"chrome\", \"firefox\", \"safari\", \"edge\"]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#iso639_1) ISO639\\_1\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nISO639_1 = Literal[\"aa\", \"ab\", \"ae\", ...]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#basicresponse) BasicResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass BasicResponse(BaseModel):\n success: bool\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#session) Session\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass Session(BaseModel):\n id: str\n team_id: str = Field(alias=\"teamId\")\n status: SessionStatus\n created_at: datetime = Field(alias=\"createdAt\")\n updated_at: datetime = Field(alias=\"updatedAt\")\n start_time: Optional[int] = Field(default=None, alias=\"startTime\")\n end_time: Optional[int] = Field(default=None, alias=\"endTime\")\n duration: Optional[int] = None\n session_url: str = Field(alias=\"sessionUrl\")\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#sessiondetail) SessionDetail\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass SessionDetail(Session):\n ws_endpoint: Optional[str] = Field(alias=\"wsEndpoint\", default=None)\n live_url: str = Field(alias=\"liveUrl\")\n token: str = Field(alias=\"token\")\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#sessionlistresponse) SessionListResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass SessionListResponse(BaseModel):\n sessions: List[Session]\n total_count: int = Field(alias=\"totalCount\")\n page: int\n per_page: int = Field(alias=\"perPage\")\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#screenconfig) ScreenConfig\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass ScreenConfig(BaseModel):\n width: int = Field(default=1280, serialization_alias=\"width\")\n height: int = Field(default=720, serialization_alias=\"height\")\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#createsessionparams) CreateSessionParams\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass CreateSessionParams(BaseModel):\n use_stealth: bool = Field(default=False, serialization_alias=\"useStealth\")\n use_proxy: bool = Field(default=False, serialization_alias=\"useProxy\")\n proxy_server: Optional[str] = Field(default=None, serialization_alias=\"proxyServer\")\n proxy_server_password: Optional[str] = Field(\n default=None, serialization_alias=\"proxyServerPassword\"\n )\n proxy_server_username: Optional[str] = Field(\n default=None, serialization_alias=\"proxyServerUsername\"\n )\n proxy_country: Optional[Country] = Field(\n default=\"US\", serialization_alias=\"proxyCountry\"\n )\n operating_systems: Optional[List[OperatingSystem]] = Field(\n default=None, serialization_alias=\"operatingSystems\"\n )\n device: Optional[List[Literal[\"desktop\", \"mobile\"]]] = Field(default=None)\n platform: Optional[List[Platform]] = Field(default=None)\n locales: List[ISO639_1] = Field(default=[\"en\"])\n screen: Optional[ScreenConfig] = Field(default=None)\n solve_captchas: bool = Field(default=False, serialization_alias=\"solveCaptchas\")\n adblock: bool = Field(default=False, serialization_alias=\"adblock\")\n trackers: bool = Field(default=False, serialization_alias=\"trackers\")\n annoyances: bool = Field(default=False, serialization_alias=\"annoyances\")\n enable_web_recording: Optional[bool] = Field(\n default=True, serialization_alias=\"enableWebRecording\"\n )\n extension_ids: Optional[List[str]] = Field(\n default=None, serialization_alias=\"extensionIds\"\n )\n accept_cookies: Optional[bool] = Field(\n default=None, serialization_alias=\"acceptCookies\"\n )\n url_blocklist: Optional[List[str]] = Field(default=None, serialization_alias=\"urlBlocklist\")\n browser_args: Optional[List[str]] = Field(default=None, serialization_alias=\"browserArgs\")\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions\\#sessionrecording) SessionRecording\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass SessionRecording(BaseModel):\n type: int\n data: Any\n timestamp: int\n delay: Optional[int] = None\n```\n\n[PreviousPython](https://www.docs.hyperbrowser.ai/reference/sdks/python) [NextProfiles](https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles)\n\nLast updated 22 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/python/sessions", "title": "Sessions | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/LiglIoCjQNiYRFqAr9k8", "og:title": "Sessions | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/LiglIoCjQNiYRFqAr9k8", "twitter:title": "Sessions | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/python/sessions", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions", "status": "completed", "error": null }, "summary": "The page discusses managing sessions in Hyperbrowser for effective web scraping and automation tasks." }, { "pathname": "/reference/sdks/python/profiles", "data": { "markdown": "Profiles \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles\\#create-session) Create Profile\n\nCreates a new profile.\n\n**Method:** `client.profiles.create(): CreateProfileResponse`\n\n**Endpoint:** `POST /api/profile`\n\n**Response:** [`CreateProfileResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles#createprofileresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nprofile = client.profiles.create()\nprint(profile.id)\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles\\#create-session-1) Get Profile\n\nGet details of an existing profile.\n\n**Method:** `client.profiles.get(id: str): ProfileResponse`\n\n**Endpoint:** `GET /api/profile/{id}`\n\n**Parameters:**\n\n- `id: string` \\- Profile ID\n\n\n**Response:** [`ProfileResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles#profileresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nprofile = client.profiles.get(\"36946080-9b81-4288-81d5-b15f2191f222\")\nprint(profile.id)\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles\\#create-session-2) Delete Profile\n\nDelete an existing profile.\n\n**Method:** `client.profiles.delete(id: str): BasicResponse`\n\n**Endpoint:** `DELETE /api/profile/{id}`\n\n**Parameters:**\n\n- `id: string` \\- Profile ID\n\n\n**Response:** [`BasicResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#basicresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nresponse = client.profiles.delete(\"36946080-9b81-4288-81d5-b15f2191f222\")\nprint(response)\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles\\#types) Types\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles\\#createprofileresponse) CreateProfileResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass CreateProfileResponse(BaseModel):\n id: str\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles\\#profileresponse) ProfileResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass ProfileResponse(BaseModel):\n id: str\n team_id: str = Field(alias=\"teamId\")\n created_at: datetime = Field(alias=\"createdAt\")\n updated_at: datetime = Field(alias=\"updatedAt\")\n```\n\n[PreviousSessions](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions) [NextScrape](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/python/profiles", "title": "Profiles | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/NIVcasQCduVokGYXRJqf", "og:title": "Profiles | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/NIVcasQCduVokGYXRJqf", "twitter:title": "Profiles | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/python/profiles", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles", "status": "completed", "error": null }, "summary": "The \"Profiles\" page in Hyperbrowser documentation explains how to manage and use user profiles for web scraping tasks." }, { "pathname": "/reference/sdks/python/scrape", "data": { "markdown": "Scrape \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape\\#start-scrape-job) Start Scrape Job\n\nStarts a scrape job for a given URL.\n\n**Method:** `client.scrape.start(params: StartScrapeJobParams): StartScrapeJobResponse`\n\n**Endpoint:** `POST /api/scrape`\n\n**Parameters:**\n\n- `StartScrapeJobParams`:\n\n\n\n- `url: string` \\- URL to scrape\n\n- `session_options?: ` [`CreateSessionParams`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#createsessionparams)\n\n- `scrape_options?: ` [`ScrapeOptions`](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape#scrapeoptions)\n\n\n**Response:** [`StartScrapeJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape#startscrapejobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nresponse = client.scrape.start(StartScrapeJobParams(url=\"https://example.com\"))\nprint(response.jobId)\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape\\#get-scrape-job) [鈥媇(https://docs.hyperbrowser.ai/sdks/node\\#get-scrape-job) Get Scrape Job\n\nRetrieves details of a specific scrape job.\n\n**Method:** `client.scrape.get(id: str): ScrapeJobResponse`\n\n**Endpoint:** `GET /api/scrape/{id}`\n\n**Parameters:**\n\n- `id: string` \\- Scrape job ID\n\n\n**Response:** [`ScrapeJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape#scrapejobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nresponse = client.scrape.get(\n \"182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e\"\n)\nprint(response.status)\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape\\#start-scrape-job-and-wait) Start Scrape Job and Wait\n\nStart a scrape job and wait for it to complete\n\n**Method**: `client.scrape.start_and_wait(params: StartScrapeJobParams): ScrapeJobResponse`\n\n**Parameters:**\n\n- `StartScrapeJobParams`:\n\n\n\n- `url: string` \\- URL to scrape\n\n- `session_options?: ` [`CreateSessionParams`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#createsessionparams)\n\n- `scrape_options?: ` [`ScrapeOptions`](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape#scrapeoptions)\n\n\n**Response:** [`ScrapeJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape#scrapejobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nresponse = client.scrape.start_and_wait(StartScrapeJobParams(url=\"https://example.com\"))\nprint(response.status)\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape\\#types) Types\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape\\#scrapeformat) ScrapeFormat\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nScrapeFormat = Literal[\"markdown\", \"html\", \"links\", \"screenshot\"]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape\\#scrapejobstatus) ScrapeJobStatus\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nScrapeJobStatus = Literal[\"pending\", \"running\", \"completed\", \"failed\"]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape\\#scrapeoptions) ScrapeOptions\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass ScrapeOptions(BaseModel):\n formats: Optional[List[ScrapeFormat]] = None\n include_tags: Optional[List[str]] = Field(\n default=None, serialization_alias=\"includeTags\"\n )\n exclude_tags: Optional[List[str]] = Field(\n default=None, serialization_alias=\"excludeTags\"\n )\n only_main_content: Optional[bool] = Field(\n default=None, serialization_alias=\"onlyMainContent\"\n )\n wait_for: Optional[int] = Field(default=None, serialization_alias=\"waitFor\")\n timeout: Optional[int] = Field(default=None, serialization_alias=\"timeout\")\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape\\#startscrapejobresponse) StartScrapeJobResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass StartScrapeJobResponse(BaseModel):\n job_id: str = Field(alias=\"jobId\")\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape\\#scrapejobdata) ScrapeJobData\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass ScrapeJobData(BaseModel):\n metadata: Optional[dict[str, Union[str, list[str]]]] = None\n html: Optional[str] = None\n markdown: Optional[str] = None\n links: Optional[List[str]] = None\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape\\#scrapejobresponse) ScrapeJobResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass ScrapeJobResponse(BaseModel):\n job_id: str = Field(alias=\"jobId\")\n status: ScrapeJobStatus\n error: Optional[str] = None\n data: Optional[ScrapeJobData] = None\n```\n\n[PreviousProfiles](https://www.docs.hyperbrowser.ai/reference/sdks/python/profiles) [NextCrawl](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/python/scrape", "title": "Scrape | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/UxBkzX14qm1xYVWOXzxt", "og:title": "Scrape | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/UxBkzX14qm1xYVWOXzxt", "twitter:title": "Scrape | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/python/scrape", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape", "status": "completed", "error": null }, "summary": "The page explains how to use Hyperbrowser for web scraping tasks." }, { "pathname": "/reference/sdks/python/crawl", "data": { "markdown": "Crawl \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl\\#start-scrape-job) Start Crawl Job\n\nStarts a crawl job for a given URL.\n\n**Method:** `client.crawl.start(params: StartCrawlJobParams): StartCrawlJobResponse`\n\n**Endpoint:** `POST /api/crawl`\n\n**Parameters:**\n\n- `StartCrawlJobParams`:\n\n\n\n- `url: string` \\- URL to scrape\n\n- `max_pages?: number` \\- Max number of pages to crawl\n\n- `follow_links?: boolean` \\- Follow links on the page\n\n- `ignore_sitemap?: boolean` \\- Ignore sitemap when finding links to crawl\n\n- `exclude_patterns?: string[]` \\- Patterns for paths to exclude from crawl\n\n- `include_patterns?: string[]` \\- Patterns for paths to include in the crawl\n\n- `session_options?: ` [`CreateSessionParams`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#createsessionparams)\n\n- `scrape_options?: ` [`ScrapeOptions`](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape#scrapeoptions)\n\n\n**Response:** [`StartCrawlJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl#startcrawljobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nresponse = client.crawl.start(StartCrawlJobParams(url=\"https://example.com\"))\nprint(response.status)\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl\\#get-scrape-job) [鈥媇(https://docs.hyperbrowser.ai/sdks/node\\#get-scrape-job) Get Crawl Job\n\nRetrieves details of a specific crawl job.\n\n**Method:** `client.crawl.get(id: str): CrawlJobResponse`\n\n**Endpoint:** `GET /api/crawl/{id}`\n\n**Parameters:**\n\n- `id: string` \\- Crawl job ID\n\n\n**Response:** [`CrawlJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl#crawljobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nresponse = client.crawl.get(\n \"182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e\"\n)\nprint(response.status)\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl\\#start-crawl-job-and-wait) Start Crawl Job and Wait\n\nStart a crawl job and wait for it to complete\n\n**Method**: `client.crawl.start_and_wait(params: StartCrawlJobParams): CrawlJobResponse`\n\n**Parameters:**\n\n- `StartCrawlJobParams`:\n\n\n\n- `url: string` \\- URL to scrape\n\n- `max_pages?: number` \\- Max number of pages to crawl\n\n- `follow_links?: boolean` \\- Follow links on the page\n\n- `ignore_sitemap?: boolean` \\- Ignore sitemap when finding links to crawl\n\n- `exclude_patterns?: string[]` \\- Patterns for paths to exclude from crawl\n\n- `include_patterns?: string[]` \\- Patterns for paths to include in the crawl\n\n- `session_options?: ` [`CreateSessionParams`](https://www.docs.hyperbrowser.ai/reference/sdks/python/sessions#createsessionparams)\n\n- `scrape_options?: ` [`ScrapeOptions`](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape#scrapeoptions)\n\n\n**Response:** [`CrawlJobResponse`](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl#crawljobresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nresponse = client.crawl.start_and_wait(StartCrawlJobParams(url=\"https://example.com\"))\nprint(response.status)\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl\\#types) Types\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl\\#crawlpagestatus) CrawlPageStatus\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nCrawlPageStatus = Literal[\"completed\", \"failed\"]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl\\#crawljobstatus) CrawlJobStatus\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nCrawlJobStatus = Literal[\"pending\", \"running\", \"completed\", \"failed\"]\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl\\#startcrawljobresponse) StartCrawlJobResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass StartCrawlJobResponse(BaseModel):\n job_id: str = Field(alias=\"jobId\")\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl\\#crawledpage) CrawledPage\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass CrawledPage(BaseModel):\n metadata: Optional[dict[str, Union[str, list[str]]]] = None\n html: Optional[str] = None\n markdown: Optional[str] = None\n links: Optional[List[str]] = None\n url: str\n status: CrawlPageStatus\n error: Optional[str] = None\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl\\#crawljobresponse) CrawlJobResponse\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass CrawlJobResponse(BaseModel):\n job_id: str = Field(alias=\"jobId\")\n status: CrawlJobStatus\n error: Optional[str] = None\n data: List[CrawledPage] = Field(alias=\"data\")\n total_crawled_pages: int = Field(alias=\"totalCrawledPages\")\n total_page_batches: int = Field(alias=\"totalPageBatches\")\n current_page_batch: int = Field(alias=\"currentPageBatch\")\n batch_size: int = Field(alias=\"batchSize\")\n```\n\n[PreviousScrape](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape) [NextExtensions](https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/python/crawl", "title": "Crawl | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/eeo0h0lBa0ADHmd6F9zA", "og:title": "Crawl | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/eeo0h0lBa0ADHmd6F9zA", "twitter:title": "Crawl | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/python/crawl", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl", "status": "completed", "error": null }, "summary": "The Crawl section of Hyperbrowser鈥檚 documentation explains web scraping techniques and automation processes." }, { "pathname": "/reference/sdks/python/extensions", "data": { "markdown": "Extensions \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions\\#add-an-extension) Add an extension\n\nAdds a new chrome **manifest V3** extension\n\n**Method:** `client.extensions.create(params:` CreateExtensionParams `): Promise<` ExtensionResponse `>`\n\n**Endpoint:** `POST /api/extensions/add`\n\n**Parameters:**\n\n- params: [**CreateExtensionParams**](https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions#createextensionparams)\n\n\n\n- `file_path: string` \\- Path to the zip containing the manifest V3 compliant extension\n\n- `name?: string` \\- Optional name to give to the extension. Does not affect functionality.\n\n\n**Response:** [**ExtensionResponse**](https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions#extensionresponse)\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nextension = client.extensions.create(\n params=CreateExtensionParams(file_path=\"/Users/test-user/Downloads/extension.zip\")\n)\nprint(extension);\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions\\#list-all-available-extension) List all available extension\n\nList all available extensions\n\n**Method:** `client.extensions.list(): Promise< ListExtensionsResponse >`\n\n**Endpoint:** `POST /api/extensions/list`\n\n**Parameters:**\n\n- N/A\n\n\n**Response:** [**List\\[ExtensionResponse**](https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions#extensionresponse) **\\]**\n\n**Example:**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nextensionsList = client.extensions.list();\nfor i in range(len(extensionsList)):\n extension = extensionsList[i]\n print(f\"extension-{i} => \", json.dumps(extension))\n\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions\\#types) Types\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions\\#createextensionparams) CreateExtensionParams\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass CreateExtensionParams(BaseModel):\n name: Optional[str] = Field(default=None, serialization_alias=\"name\")\n file_path: str = Field(serialization_alias=\"filePath\")\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions\\#extensionresponse) **ExtensionResponse**\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nclass ExtensionResponse(BaseModel):\n id: str = Field(serialization_alias=\"id\")\n name: str = Field(serialization_alias=\"name\")\n created_at: datetime = Field(serialization_alias=\"createdAt\",alias=\"createdAt\")\n updated_at: datetime = Field(serialization_alias=\"updatedAt\",alias=\"updatedAt\")\n```\n\n[PreviousCrawl](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl) [NextAPI Reference](https://www.docs.hyperbrowser.ai/reference/api-reference)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/sdks/python/extensions", "title": "Extensions | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/DuClab7mJI4zReEGWLZA", "og:title": "Extensions | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/DuClab7mJI4zReEGWLZA", "twitter:title": "Extensions | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/sdks/python/extensions", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions", "status": "completed", "error": null }, "summary": "Explore Hyperbrowser extensions for enhanced web scraping and automation capabilities in your projects." }, { "pathname": "/reference/api-reference", "data": { "markdown": "API Reference \\| Hyperbrowser\n\nHyperbrowser API endpoints are documented using OpenAPI specification. It offers methods to create new sessions, get session details, stop sessions, and more.\n\n[PreviousExtensions](https://www.docs.hyperbrowser.ai/reference/sdks/python/extensions) [NextSessions](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference", "title": "API Reference | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/jfjr2o57t88W7vUOHMDo", "og:title": "API Reference | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/jfjr2o57t88W7vUOHMDo", "twitter:title": "API Reference | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference", "status": "completed", "error": null }, "summary": "API Reference for Hyperbrowser provides detailed information on using its web scraping and automation features." }, { "pathname": "/reference/api-reference/sessions", "data": { "markdown": "Sessions \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions\\#api-session) Create new session\n\npost\n\nhttps://app.hyperbrowser.ai/api/session\n\nAuthorizations\n\nBody\n\nuseStealthboolean 路 default: false\n\nuseProxyboolean 路 default: false\n\nproxyServerstring\n\nproxyServerPasswordstring\n\nproxyServerUsernamestring\n\nproxyCountrystring 路 enum 路 default: \"US\"\n\nOptions: `US`, `GB`, `CA`\n\nproxyStatestring 路 enum \\| nullable\n\nOptional state code for proxies to US states. Is mutually exclusive with proxyCity. Takes in two letter state code.\n\nOptions: `AL`, `AK`, `AZ`\n\nproxyCitystring \\| nullable\n\nDesired Country. Is mutually exclusive with proxyState. Some cities might not be supported, so before using a new city, we recommend trying it out\n\nExample: `new york`\n\noperatingSystemsstring 路 enum\\[\\]\n\n### Show child attributes\n\ndevicestring 路 enum\\[\\]\n\n### Show child attributes\n\nplatformstring 路 enum\\[\\]\n\n### Show child attributes\n\nlocalesstring 路 enum\\[\\] 路 default: \\[\"en\"\\]\n\n### Show child attributes\n\nscreenobject\n\n### Show child attributes\n\nsolveCaptchasboolean 路 default: false\n\nadblockboolean 路 default: false\n\ntrackersboolean 路 default: false\n\nannoyancesboolean 路 default: false\n\nenableWebRecordingboolean\n\nprofileobject\n\n### Show child attributes\n\nacceptCookiesboolean\n\nextensionIdsstring 路 uuid\\[\\] 路 default: \\[\\]\n\nurlBlockliststring\\[\\] 路 default: \\[\\]\n\nbrowserArgsstring\\[\\] 路 default: \\[\\]\n\nResponses\n\n200\n\nSession created\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request POST \\\n --url 'https://app.hyperbrowser.ai/api/session' \\\n --header 'x-api-key: YOUR_API_KEY' \\\n --header 'Content-Type: application/json' \\\n --data '{\n \"useStealth\": true,\n \"useProxy\": true,\n \"proxyServer\": \"text\",\n \"proxyServerPassword\": \"text\",\n \"proxyServerUsername\": \"text\",\n \"proxyCountry\": \"US\",\n \"proxyState\": \"AL\",\n \"proxyCity\": \"new york\",\n \"operatingSystems\": [\\\n \"windows\"\\\n ],\n \"device\": [\\\n \"desktop\"\\\n ],\n \"platform\": [\\\n \"chrome\"\\\n ],\n \"locales\": [\\\n \"en\"\\\n ],\n \"screen\": {\n \"width\": 1,\n \"height\": 1\n },\n \"solveCaptchas\": true,\n \"adblock\": true,\n \"trackers\": true,\n \"annoyances\": true,\n \"enableWebRecording\": true,\n \"profile\": {\n \"id\": \"text\",\n \"persistChanges\": true\n },\n \"acceptCookies\": true,\n \"extensionIds\": [\\\n \"123e4567-e89b-12d3-a456-426614174000\"\\\n ],\n \"urlBlocklist\": [\\\n \"text\"\\\n ],\n \"browserArgs\": [\\\n \"text\"\\\n ]\n }'\n```\n\nTest it\n\n200\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"id\": \"123e4567-e89b-12d3-a456-426614174000\",\n \"teamId\": \"123e4567-e89b-12d3-a456-426614174000\",\n \"status\": \"active\",\n \"startTime\": \"text\",\n \"endTime\": \"text\",\n \"createdAt\": \"text\",\n \"updatedAt\": \"text\",\n \"sessionUrl\": \"text\",\n \"liveUrl\": \"text\",\n \"token\": \"text\",\n \"wsEndpoint\": \"text\"\n}\n```\n\nSession created\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions\\#api-session-id) Get session by ID\n\nget\n\nhttps://app.hyperbrowser.ai/api/session/{id}\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nSession details\n\napplication/json\n\n404\n\nSession not found\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/session/{id}' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"id\": \"123e4567-e89b-12d3-a456-426614174000\",\n \"teamId\": \"123e4567-e89b-12d3-a456-426614174000\",\n \"status\": \"active\",\n \"startTime\": \"text\",\n \"endTime\": \"text\",\n \"createdAt\": \"text\",\n \"updatedAt\": \"text\",\n \"sessionUrl\": \"text\",\n \"liveUrl\": \"text\",\n \"token\": \"text\",\n \"wsEndpoint\": \"text\"\n}\n```\n\nSession details\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions\\#api-session-id-stop) Stop a session\n\nput\n\nhttps://app.hyperbrowser.ai/api/session/{id}/stop\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nSession stopped successfully\n\napplication/json\n\n404\n\nSession not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request PUT \\\n --url 'https://app.hyperbrowser.ai/api/session/{id}/stop' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"success\": true\n}\n```\n\nSession stopped successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions\\#api-sessions) Get list of sessions\n\nget\n\nhttps://app.hyperbrowser.ai/api/sessions\n\nAuthorizations\n\nQuery parameters\n\npagenumber 路 default: 1\n\nstatusstring 路 enum\n\nOptions: `active`, `closed`, `error`\n\nResponses\n\n200\n\nList of sessions\n\napplication/json\n\n400\n\nInvalid query parameters\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/sessions' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n400\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"sessions\": [\\\n {\\\n \"id\": \"123e4567-e89b-12d3-a456-426614174000\",\\\n \"teamId\": \"123e4567-e89b-12d3-a456-426614174000\",\\\n \"status\": \"active\",\\\n \"startTime\": \"text\",\\\n \"endTime\": \"text\",\\\n \"createdAt\": \"text\",\\\n \"updatedAt\": \"text\"\\\n }\\\n ],\n \"totalCount\": 100,\n \"page\": 1,\n \"pageSize\": 10\n}\n```\n\nList of sessions\n\n[PreviousAPI Reference](https://www.docs.hyperbrowser.ai/reference/api-reference) [NextScrape](https://www.docs.hyperbrowser.ai/reference/api-reference/scrape)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference/sessions", "title": "Sessions | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/A1Mh2sEbh2IrikFy5wpw", "og:title": "Sessions | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference/sessions", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/A1Mh2sEbh2IrikFy5wpw", "twitter:title": "Sessions | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference/sessions", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference/sessions", "status": "completed", "error": null }, "summary": "The \"Sessions\" page in Hyperbrowser covers managing and utilizing sessions for web scraping and automation tasks." }, { "pathname": "/reference/api-reference/crawl", "data": { "markdown": "Crawl \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/crawl\\#api-crawl) Start a crawl job\n\npost\n\nhttps://app.hyperbrowser.ai/api/crawl\n\nAuthorizations\n\nBody\n\nurlstringrequired\n\nmaxPagesinteger 路 min: 1\n\nfollowLinksboolean 路 default: true\n\nignoreSitemapboolean 路 default: false\n\nexcludePatternsstring\\[\\]\n\nincludePatternsstring\\[\\]\n\nsessionOptionsobject 路 default: {\"useStealth\":false,\"useProxy\":false,\"acceptCookies\":false}\n\n### Show child attributes\n\nscrapeOptionsobject\n\n### Show child attributes\n\nResponses\n\n200\n\nCrawl job started successfully\n\napplication/json\n\n400\n\nInvalid request parameters\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request POST \\\n --url 'https://app.hyperbrowser.ai/api/crawl' \\\n --header 'x-api-key: YOUR_API_KEY' \\\n --header 'Content-Type: application/json' \\\n --data '{\n \"url\": \"text\",\n \"maxPages\": 1,\n \"followLinks\": true,\n \"ignoreSitemap\": true,\n \"excludePatterns\": [\\\n \"text\"\\\n ],\n \"includePatterns\": [\\\n \"text\"\\\n ],\n \"sessionOptions\": {\n \"useStealth\": true,\n \"useProxy\": true,\n \"proxyServer\": \"text\",\n \"proxyServerPassword\": \"text\",\n \"proxyServerUsername\": \"text\",\n \"proxyCountry\": \"US\",\n \"proxyState\": \"AL\",\n \"proxyCity\": \"new york\",\n \"operatingSystems\": [\\\n \"windows\"\\\n ],\n \"device\": [\\\n \"desktop\"\\\n ],\n \"platform\": [\\\n \"chrome\"\\\n ],\n \"locales\": [\\\n \"en\"\\\n ],\n \"screen\": {\n \"width\": 1,\n \"height\": 1\n },\n \"solveCaptchas\": true,\n \"adblock\": true,\n \"trackers\": true,\n \"annoyances\": true,\n \"enableWebRecording\": true,\n \"profile\": {\n \"id\": \"text\",\n \"persistChanges\": true\n },\n \"acceptCookies\": true,\n \"extensionIds\": [\\\n \"123e4567-e89b-12d3-a456-426614174000\"\\\n ],\n \"urlBlocklist\": [\\\n \"text\"\\\n ],\n \"browserArgs\": [\\\n \"text\"\\\n ]\n },\n \"scrapeOptions\": {\n \"formats\": [\\\n \"html\"\\\n ],\n \"includeTags\": [\\\n \"text\"\\\n ],\n \"excludeTags\": [\\\n \"text\"\\\n ],\n \"onlyMainContent\": true,\n \"waitFor\": 1,\n \"timeout\": 1,\n \"waitUntil\": \"load\",\n \"screenshotOptions\": {\n \"fullPage\": true,\n \"format\": \"jpeg\"\n }\n }\n }'\n```\n\nTest it\n\n200\n\n400\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\"\n}\n```\n\nCrawl job started successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/crawl\\#api-crawl-id) Get crawl job status and results\n\nget\n\nhttps://app.hyperbrowser.ai/api/crawl/{id}\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nQuery parameters\n\npageinteger\n\nbatchSizeinteger 路 min: 1\n\nResponses\n\n200\n\nCrawl job details retrieved successfully\n\napplication/json\n\n404\n\nCrawl job not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/crawl/{id}' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"123e4567-e89b-12d3-a456-426614174000\",\n \"status\": \"pending\",\n \"error\": \"text\",\n \"totalCrawledPages\": 1,\n \"totalPageBatches\": 1,\n \"currentPageBatch\": 1,\n \"batchSize\": 1,\n \"data\": [\\\n {\\\n \"url\": \"text\",\\\n \"status\": \"completed\",\\\n \"error\": \"text\",\\\n \"metadata\": {\\\n \"ANY_ADDITIONAL_PROPERTY\": \"text\"\\\n },\\\n \"markdown\": \"text\",\\\n \"html\": \"text\",\\\n \"links\": [\\\n \"text\"\\\n ],\\\n \"screenshot\": \"text\"\\\n }\\\n ]\n}\n```\n\nCrawl job details retrieved successfully\n\n[PreviousScrape](https://www.docs.hyperbrowser.ai/reference/api-reference/scrape) [NextExtract](https://www.docs.hyperbrowser.ai/reference/api-reference/extract)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference/crawl", "title": "Crawl | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/FTp6nuGHlI0CVQUzMT5q", "og:title": "Crawl | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference/crawl", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/FTp6nuGHlI0CVQUzMT5q", "twitter:title": "Crawl | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference/crawl", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference/crawl", "status": "completed", "error": null }, "summary": "The \"Crawl\" section of Hyperbrowser documentation explains how to use the tool for web scraping and automation tasks." }, { "pathname": "/reference/api-reference/scrape", "data": { "markdown": "Scrape \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/scrape\\#api-scrape) Create new scrape job\n\npost\n\nhttps://app.hyperbrowser.ai/api/scrape\n\nAuthorizations\n\nBody\n\nurlstring 路 min: 1required\n\nsessionOptionsobject 路 default: {\"useStealth\":false,\"useProxy\":false,\"acceptCookies\":false}\n\n### Show child attributes\n\nscrapeOptionsobject\n\n### Show child attributes\n\nResponses\n\n200\n\nScrape job created\n\napplication/json\n\n400\n\nInvalid request parameters\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request POST \\\n --url 'https://app.hyperbrowser.ai/api/scrape' \\\n --header 'x-api-key: YOUR_API_KEY' \\\n --header 'Content-Type: application/json' \\\n --data '{\n \"url\": \"text\",\n \"sessionOptions\": {\n \"useStealth\": true,\n \"useProxy\": true,\n \"proxyServer\": \"text\",\n \"proxyServerPassword\": \"text\",\n \"proxyServerUsername\": \"text\",\n \"proxyCountry\": \"US\",\n \"proxyState\": \"AL\",\n \"proxyCity\": \"new york\",\n \"operatingSystems\": [\\\n \"windows\"\\\n ],\n \"device\": [\\\n \"desktop\"\\\n ],\n \"platform\": [\\\n \"chrome\"\\\n ],\n \"locales\": [\\\n \"en\"\\\n ],\n \"screen\": {\n \"width\": 1,\n \"height\": 1\n },\n \"solveCaptchas\": true,\n \"adblock\": true,\n \"trackers\": true,\n \"annoyances\": true,\n \"enableWebRecording\": true,\n \"profile\": {\n \"id\": \"text\",\n \"persistChanges\": true\n },\n \"acceptCookies\": true,\n \"extensionIds\": [\\\n \"123e4567-e89b-12d3-a456-426614174000\"\\\n ],\n \"urlBlocklist\": [\\\n \"text\"\\\n ],\n \"browserArgs\": [\\\n \"text\"\\\n ]\n },\n \"scrapeOptions\": {\n \"formats\": [\\\n \"html\"\\\n ],\n \"includeTags\": [\\\n \"text\"\\\n ],\n \"excludeTags\": [\\\n \"text\"\\\n ],\n \"onlyMainContent\": true,\n \"waitFor\": 1,\n \"timeout\": 1,\n \"waitUntil\": \"load\",\n \"screenshotOptions\": {\n \"fullPage\": true,\n \"format\": \"jpeg\"\n }\n }\n }'\n```\n\nTest it\n\n200\n\n400\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\"\n}\n```\n\nScrape job created\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/scrape\\#api-scrape-id) Get scrape job status and result\n\nget\n\nhttps://app.hyperbrowser.ai/api/scrape/{id}\n\nAuthorizations\n\nPath parameters\n\nidstring 路 uuidrequired\n\nResponses\n\n200\n\nScrape job details\n\napplication/json\n\n404\n\nJob not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/scrape/{id}' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\",\n \"status\": \"pending\",\n \"data\": {\n \"metadata\": {\n \"ANY_ADDITIONAL_PROPERTY\": \"text\"\n },\n \"markdown\": \"text\",\n \"html\": \"text\",\n \"links\": [\\\n \"text\"\\\n ],\n \"screenshot\": \"text\"\n },\n \"error\": \"text\"\n}\n```\n\nScrape job details\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/scrape\\#api-scrape-batch) Start a batch scrape job\n\npost\n\nhttps://app.hyperbrowser.ai/api/scrape/batch\n\nAuthorizations\n\nBody\n\nurlsstring\\[\\]required\n\nsessionOptionsobject 路 default: {\"useStealth\":false,\"useProxy\":false,\"acceptCookies\":false}\n\n### Show child attributes\n\nscrapeOptionsobject\n\n### Show child attributes\n\nResponses\n\n200\n\nBatch scrape job started successfully\n\napplication/json\n\n400\n\nInvalid request parameters\n\napplication/json\n\n402\n\nInsufficient plan\n\napplication/json\n\n429\n\nToo many concurrent batch scrape jobs\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request POST \\\n --url 'https://app.hyperbrowser.ai/api/scrape/batch' \\\n --header 'x-api-key: YOUR_API_KEY' \\\n --header 'Content-Type: application/json' \\\n --data '{\n \"urls\": [\\\n \"text\"\\\n ],\n \"sessionOptions\": {\n \"useStealth\": true,\n \"useProxy\": true,\n \"proxyServer\": \"text\",\n \"proxyServerPassword\": \"text\",\n \"proxyServerUsername\": \"text\",\n \"proxyCountry\": \"US\",\n \"proxyState\": \"AL\",\n \"proxyCity\": \"new york\",\n \"operatingSystems\": [\\\n \"windows\"\\\n ],\n \"device\": [\\\n \"desktop\"\\\n ],\n \"platform\": [\\\n \"chrome\"\\\n ],\n \"locales\": [\\\n \"en\"\\\n ],\n \"screen\": {\n \"width\": 1,\n \"height\": 1\n },\n \"solveCaptchas\": true,\n \"adblock\": true,\n \"trackers\": true,\n \"annoyances\": true,\n \"enableWebRecording\": true,\n \"profile\": {\n \"id\": \"text\",\n \"persistChanges\": true\n },\n \"acceptCookies\": true,\n \"extensionIds\": [\\\n \"123e4567-e89b-12d3-a456-426614174000\"\\\n ],\n \"urlBlocklist\": [\\\n \"text\"\\\n ],\n \"browserArgs\": [\\\n \"text\"\\\n ]\n },\n \"scrapeOptions\": {\n \"formats\": [\\\n \"html\"\\\n ],\n \"includeTags\": [\\\n \"text\"\\\n ],\n \"excludeTags\": [\\\n \"text\"\\\n ],\n \"onlyMainContent\": true,\n \"waitFor\": 1,\n \"timeout\": 1,\n \"waitUntil\": \"load\",\n \"screenshotOptions\": {\n \"fullPage\": true,\n \"format\": \"jpeg\"\n }\n }\n }'\n```\n\nTest it\n\n200\n\n400\n\n402\n\n429\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\"\n}\n```\n\nBatch scrape job started successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/scrape\\#api-scrape-batch-id) Get batch scrape job status and results\n\nget\n\nhttps://app.hyperbrowser.ai/api/scrape/batch/{id}\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nBatch scrape job details\n\napplication/json\n\n400\n\nInvalid request parameters\n\napplication/json\n\n404\n\nBatch scrape job not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/scrape/batch/{id}' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n400\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\",\n \"status\": \"pending\",\n \"data\": [\\\n {\\\n \"url\": \"text\",\\\n \"status\": \"completed\",\\\n \"error\": \"text\",\\\n \"metadata\": {\\\n \"ANY_ADDITIONAL_PROPERTY\": \"text\"\\\n },\\\n \"markdown\": \"text\",\\\n \"html\": \"text\",\\\n \"links\": [\\\n \"text\"\\\n ],\\\n \"screenshot\": \"text\"\\\n }\\\n ],\n \"error\": \"text\",\n \"totalScrapedPages\": 1,\n \"totalPageBatches\": 1,\n \"currentPageBatch\": 1,\n \"batchSize\": 1\n}\n```\n\nBatch scrape job details\n\n[PreviousSessions](https://www.docs.hyperbrowser.ai/reference/api-reference/sessions) [NextCrawl](https://www.docs.hyperbrowser.ai/reference/api-reference/crawl)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference/scrape", "title": "Scrape | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/aFdFgfoHmi9hzQOIvyt8", "og:title": "Scrape | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference/scrape", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/aFdFgfoHmi9hzQOIvyt8", "twitter:title": "Scrape | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference/scrape", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference/scrape", "status": "completed", "error": null }, "summary": "The \"Scrape\" section of Hyperbrowser documentation explains web scraping techniques and automation features of the tool." }, { "pathname": "/reference/api-reference/extract", "data": { "markdown": "Extract \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/extract\\#api-extract) Start an extract job\n\npost\n\nhttps://app.hyperbrowser.ai/api/extract\n\nAuthorizations\n\nBody\n\nurlsstring\\[\\]required\n\nsystemPromptstring\n\nmaxLinksnumber\n\npromptstring\n\nschemaobject\n\nsessionOptionsobject 路 default: {\"useStealth\":false,\"useProxy\":false,\"acceptCookies\":false}\n\n### Show child attributes\n\nwaitFornumber 路 default: 0\n\nResponses\n\n200\n\nExtract job started successfully\n\napplication/json\n\n400\n\nInvalid request parameters\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request POST \\\n --url 'https://app.hyperbrowser.ai/api/extract' \\\n --header 'x-api-key: YOUR_API_KEY' \\\n --header 'Content-Type: application/json' \\\n --data '{\n \"urls\": [\\\n \"text\"\\\n ],\n \"systemPrompt\": \"text\",\n \"maxLinks\": 1,\n \"prompt\": \"text\",\n \"schema\": {},\n \"sessionOptions\": {\n \"useStealth\": true,\n \"useProxy\": true,\n \"proxyServer\": \"text\",\n \"proxyServerPassword\": \"text\",\n \"proxyServerUsername\": \"text\",\n \"proxyCountry\": \"US\",\n \"proxyState\": \"AL\",\n \"proxyCity\": \"new york\",\n \"operatingSystems\": [\\\n \"windows\"\\\n ],\n \"device\": [\\\n \"desktop\"\\\n ],\n \"platform\": [\\\n \"chrome\"\\\n ],\n \"locales\": [\\\n \"en\"\\\n ],\n \"screen\": {\n \"width\": 1,\n \"height\": 1\n },\n \"solveCaptchas\": true,\n \"adblock\": true,\n \"trackers\": true,\n \"annoyances\": true,\n \"enableWebRecording\": true,\n \"profile\": {\n \"id\": \"text\",\n \"persistChanges\": true\n },\n \"acceptCookies\": true,\n \"extensionIds\": [\\\n \"123e4567-e89b-12d3-a456-426614174000\"\\\n ],\n \"urlBlocklist\": [\\\n \"text\"\\\n ],\n \"browserArgs\": [\\\n \"text\"\\\n ]\n },\n \"waitFor\": 1\n }'\n```\n\nTest it\n\n200\n\n400\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\"\n}\n```\n\nExtract job started successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/extract\\#api-extract-id) Get extract job status and results\n\nget\n\nhttps://app.hyperbrowser.ai/api/extract/{id}\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nExtract job details\n\napplication/json\n\n404\n\nExtract job not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/extract/{id}' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\",\n \"status\": \"pending\",\n \"data\": {},\n \"error\": \"text\"\n}\n```\n\nExtract job details\n\n[PreviousCrawl](https://www.docs.hyperbrowser.ai/reference/api-reference/crawl) [NextAgents](https://www.docs.hyperbrowser.ai/reference/api-reference/agents)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference/extract", "title": "Extract | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/c23V0bA6lE6DToWa3lW7", "og:title": "Extract | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference/extract", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/c23V0bA6lE6DToWa3lW7", "twitter:title": "Extract | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference/extract", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference/extract", "status": "completed", "error": null }, "summary": "The Extract page of Hyperbrowser provides guidelines on data extraction methods and tools for web scraping." }, { "pathname": "/reference/api-reference/agents", "data": { "markdown": "Agents \\| Hyperbrowser\n\nAPI Reference for supported agents for Hyperbrowser.\n\n[PreviousExtract](https://www.docs.hyperbrowser.ai/reference/api-reference/extract) [NextBrowser Use](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use)\n\nLast updated 16 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference/agents", "title": "Agents | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/4pamXgTXfUJt8TAKOjPT", "og:title": "Agents | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference/agents", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/4pamXgTXfUJt8TAKOjPT", "twitter:title": "Agents | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference/agents", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference/agents", "status": "completed", "error": null }, "summary": "The \"Agents\" page in Hyperbrowser documentation discusses automated entities for web scraping and task execution." }, { "pathname": "/reference/api-reference/agents/browser-use", "data": { "markdown": "Browser Use \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use\\#api-task-browser-use) Start a browser use task\n\npost\n\nhttps://app.hyperbrowser.ai/api/task/browser-use\n\nAuthorizations\n\nBody\n\ntaskstringrequired\n\nllmstring 路 enum 路 default: \"gemini-2.0-flash\"\n\nOptions: `gpt-4o`, `gpt-4o-mini`, `claude-3-7-sonnet-20250219`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`, `gemini-2.0-flash`\n\nsessionIdstring\n\nvalidateOutputboolean 路 default: false\n\nuseVisionboolean 路 default: true\n\nuseVisionForPlannerboolean 路 default: false\n\nmaxActionsPerStepnumber 路 default: 10\n\nmaxInputTokensnumber 路 default: 128000\n\nplannerLlmstring 路 enum 路 default: \"gemini-2.0-flash\"\n\nOptions: `gpt-4o`, `gpt-4o-mini`, `claude-3-7-sonnet-20250219`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`, `gemini-2.0-flash`\n\npageExtractionLlmstring 路 enum 路 default: \"gemini-2.0-flash\"\n\nOptions: `gpt-4o`, `gpt-4o-mini`, `claude-3-7-sonnet-20250219`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`, `gemini-2.0-flash`\n\nplannerIntervalnumber 路 default: 10\n\nmaxStepsnumber 路 default: 20\n\nkeepBrowserOpenboolean 路 default: false\n\nsessionOptionsobject 路 default: {\"useStealth\":false,\"useProxy\":false,\"acceptCookies\":false}\n\n### Show child attributes\n\nResponses\n\n200\n\nBrowser use task started successfully\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request POST \\\n --url 'https://app.hyperbrowser.ai/api/task/browser-use' \\\n --header 'x-api-key: YOUR_API_KEY' \\\n --header 'Content-Type: application/json' \\\n --data '{\n \"task\": \"text\",\n \"llm\": \"gpt-4o\",\n \"sessionId\": \"text\",\n \"validateOutput\": true,\n \"useVision\": true,\n \"useVisionForPlanner\": true,\n \"maxActionsPerStep\": 1,\n \"maxInputTokens\": 1,\n \"plannerLlm\": \"gpt-4o\",\n \"pageExtractionLlm\": \"gpt-4o\",\n \"plannerInterval\": 1,\n \"maxSteps\": 1,\n \"keepBrowserOpen\": true,\n \"sessionOptions\": {\n \"useStealth\": true,\n \"useProxy\": true,\n \"proxyServer\": \"text\",\n \"proxyServerPassword\": \"text\",\n \"proxyServerUsername\": \"text\",\n \"proxyCountry\": \"US\",\n \"proxyState\": \"AL\",\n \"proxyCity\": \"new york\",\n \"operatingSystems\": [\\\n \"windows\"\\\n ],\n \"device\": [\\\n \"desktop\"\\\n ],\n \"platform\": [\\\n \"chrome\"\\\n ],\n \"locales\": [\\\n \"en\"\\\n ],\n \"screen\": {\n \"width\": 1,\n \"height\": 1\n },\n \"solveCaptchas\": true,\n \"adblock\": true,\n \"trackers\": true,\n \"annoyances\": true,\n \"enableWebRecording\": true,\n \"profile\": {\n \"id\": \"text\",\n \"persistChanges\": true\n },\n \"acceptCookies\": true,\n \"extensionIds\": [\\\n \"123e4567-e89b-12d3-a456-426614174000\"\\\n ],\n \"urlBlocklist\": [\\\n \"text\"\\\n ],\n \"browserArgs\": [\\\n \"text\"\\\n ]\n }\n }'\n```\n\nTest it\n\n200\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\",\n \"liveUrl\": \"text\"\n}\n```\n\nBrowser use task started successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use\\#api-task-browser-use-id) Get browser use task status and results\n\nget\n\nhttps://app.hyperbrowser.ai/api/task/browser-use/{id}\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nBrowser use task details\n\napplication/json\n\n404\n\nBrowser use task not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/task/browser-use/{id}' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\",\n \"status\": \"pending\",\n \"data\": {\n \"steps\": [\\\n {}\\\n ],\n \"finalResult\": \"text\"\n },\n \"error\": \"text\",\n \"liveUrl\": \"text\"\n}\n```\n\nBrowser use task details\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use\\#api-task-browser-use-id-stop) Stop a browser use task\n\nput\n\nhttps://app.hyperbrowser.ai/api/task/browser-use/{id}/stop\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nBrowser use task stopped successfully\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request PUT \\\n --url 'https://app.hyperbrowser.ai/api/task/browser-use/{id}/stop' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"success\": true\n}\n```\n\nBrowser use task stopped successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use\\#api-task-browser-use-id-status) Get browser use task status\n\nget\n\nhttps://app.hyperbrowser.ai/api/task/browser-use/{id}/status\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nBrowser use task status\n\napplication/json\n\n404\n\nBrowser use task not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/task/browser-use/{id}/status' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"status\": \"pending\"\n}\n```\n\nBrowser use task status\n\n[PreviousAgents](https://www.docs.hyperbrowser.ai/reference/api-reference/agents) [NextClaude Computer Use](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use)\n\nLast updated 16 days ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference/agents/browser-use", "title": "Browser Use | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/jvSsQufWvYV5ZXNnQmct", "og:title": "Browser Use | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/jvSsQufWvYV5ZXNnQmct", "twitter:title": "Browser Use | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference/agents/browser-use", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use", "status": "completed", "error": null }, "summary": "The page explains how to effectively utilize browser features in Hyperbrowser for web scraping and automation tasks." }, { "pathname": "/reference/api-reference/agents/claude-computer-use", "data": { "markdown": "Claude Computer Use \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use\\#api-task-claude-computer-use) Start a claude computer use task\n\npost\n\nhttps://app.hyperbrowser.ai/api/task/claude-computer-use\n\nAuthorizations\n\nBody\n\ntaskstringrequired\n\nsessionIdstring\n\nmaxFailuresnumber 路 default: 3\n\nmaxStepsnumber 路 default: 20\n\nkeepBrowserOpenboolean 路 default: false\n\nsessionOptionsobject 路 default: {\"useStealth\":false,\"useProxy\":false,\"acceptCookies\":false}\n\n### Show child attributes\n\nResponses\n\n200\n\nClaude computer use task started successfully\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request POST \\\n --url 'https://app.hyperbrowser.ai/api/task/claude-computer-use' \\\n --header 'x-api-key: YOUR_API_KEY' \\\n --header 'Content-Type: application/json' \\\n --data '{\n \"task\": \"text\",\n \"sessionId\": \"text\",\n \"maxFailures\": 1,\n \"maxSteps\": 1,\n \"keepBrowserOpen\": true,\n \"sessionOptions\": {\n \"useStealth\": true,\n \"useProxy\": true,\n \"proxyServer\": \"text\",\n \"proxyServerPassword\": \"text\",\n \"proxyServerUsername\": \"text\",\n \"proxyCountry\": \"US\",\n \"proxyState\": \"AL\",\n \"proxyCity\": \"new york\",\n \"operatingSystems\": [\\\n \"windows\"\\\n ],\n \"device\": [\\\n \"desktop\"\\\n ],\n \"platform\": [\\\n \"chrome\"\\\n ],\n \"locales\": [\\\n \"en\"\\\n ],\n \"screen\": {\n \"width\": 1,\n \"height\": 1\n },\n \"solveCaptchas\": true,\n \"adblock\": true,\n \"trackers\": true,\n \"annoyances\": true,\n \"enableWebRecording\": true,\n \"profile\": {\n \"id\": \"text\",\n \"persistChanges\": true\n },\n \"acceptCookies\": true,\n \"extensionIds\": [\\\n \"123e4567-e89b-12d3-a456-426614174000\"\\\n ],\n \"urlBlocklist\": [\\\n \"text\"\\\n ],\n \"browserArgs\": [\\\n \"text\"\\\n ]\n }\n }'\n```\n\nTest it\n\n200\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\",\n \"liveUrl\": \"text\"\n}\n```\n\nClaude computer use task started successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use\\#api-task-claude-computer-use-id) Get claude computer use task status and results\n\nget\n\nhttps://app.hyperbrowser.ai/api/task/claude-computer-use/{id}\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nClaude computer use task details\n\napplication/json\n\n404\n\nClaude computer use task not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/task/claude-computer-use/{id}' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\",\n \"status\": \"pending\",\n \"data\": {\n \"steps\": [\\\n {}\\\n ],\n \"finalResult\": \"text\"\n },\n \"error\": \"text\",\n \"liveUrl\": \"text\"\n}\n```\n\nClaude computer use task details\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use\\#api-task-claude-computer-use-id-stop) Stop a claude computer use task\n\nput\n\nhttps://app.hyperbrowser.ai/api/task/claude-computer-use/{id}/stop\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nClaude computer use task stopped successfully\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request PUT \\\n --url 'https://app.hyperbrowser.ai/api/task/claude-computer-use/{id}/stop' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"success\": true\n}\n```\n\nClaude computer use task stopped successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use\\#api-task-claude-computer-use-id-status) Get claude computer use task status\n\nget\n\nhttps://app.hyperbrowser.ai/api/task/claude-computer-use/{id}/status\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nClaude computer use task status\n\napplication/json\n\n404\n\nClaude computer use task not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/task/claude-computer-use/{id}/status' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"status\": \"pending\"\n}\n```\n\nClaude computer use task status\n\n[PreviousBrowser Use](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/browser-use) [NextOpenAI CUA](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua)\n\nLast updated 16 hours ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use", "title": "Claude Computer Use | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/G6ZpZfxqd1Jh7rPiVtWg", "og:title": "Claude Computer Use | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/G6ZpZfxqd1Jh7rPiVtWg", "twitter:title": "Claude Computer Use | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use", "status": "completed", "error": null }, "summary": "The page discusses using Claude for web scraping and automation with Hyperbrowser tools." }, { "pathname": "/reference/api-reference/agents/openai-cua", "data": { "markdown": "OpenAI CUA \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua\\#api-task-cua) Start a CUA task\n\npost\n\nhttps://app.hyperbrowser.ai/api/task/cua\n\nAuthorizations\n\nBody\n\ntaskstringrequired\n\nsessionIdstring\n\nmaxFailuresnumber 路 default: 3\n\nmaxStepsnumber 路 default: 20\n\nkeepBrowserOpenboolean 路 default: false\n\nsessionOptionsobject 路 default: {\"useStealth\":false,\"useProxy\":false,\"acceptCookies\":false}\n\n### Show child attributes\n\nResponses\n\n200\n\nCUA task started successfully\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request POST \\\n --url 'https://app.hyperbrowser.ai/api/task/cua' \\\n --header 'x-api-key: YOUR_API_KEY' \\\n --header 'Content-Type: application/json' \\\n --data '{\n \"task\": \"text\",\n \"sessionId\": \"text\",\n \"maxFailures\": 1,\n \"maxSteps\": 1,\n \"keepBrowserOpen\": true,\n \"sessionOptions\": {\n \"useStealth\": true,\n \"useProxy\": true,\n \"proxyServer\": \"text\",\n \"proxyServerPassword\": \"text\",\n \"proxyServerUsername\": \"text\",\n \"proxyCountry\": \"US\",\n \"proxyState\": \"AL\",\n \"proxyCity\": \"new york\",\n \"operatingSystems\": [\\\n \"windows\"\\\n ],\n \"device\": [\\\n \"desktop\"\\\n ],\n \"platform\": [\\\n \"chrome\"\\\n ],\n \"locales\": [\\\n \"en\"\\\n ],\n \"screen\": {\n \"width\": 1,\n \"height\": 1\n },\n \"solveCaptchas\": true,\n \"adblock\": true,\n \"trackers\": true,\n \"annoyances\": true,\n \"enableWebRecording\": true,\n \"profile\": {\n \"id\": \"text\",\n \"persistChanges\": true\n },\n \"acceptCookies\": true,\n \"extensionIds\": [\\\n \"123e4567-e89b-12d3-a456-426614174000\"\\\n ],\n \"urlBlocklist\": [\\\n \"text\"\\\n ],\n \"browserArgs\": [\\\n \"text\"\\\n ]\n }\n }'\n```\n\nTest it\n\n200\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\",\n \"liveUrl\": \"text\"\n}\n```\n\nCUA task started successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua\\#api-task-cua-id) Get CUA task status and results\n\nget\n\nhttps://app.hyperbrowser.ai/api/task/cua/{id}\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nCUA task details\n\napplication/json\n\n404\n\nCUA task not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/task/cua/{id}' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"jobId\": \"text\",\n \"status\": \"pending\",\n \"data\": {\n \"steps\": [\\\n {}\\\n ],\n \"finalResult\": \"text\"\n },\n \"error\": \"text\",\n \"liveUrl\": \"text\"\n}\n```\n\nCUA task details\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua\\#api-task-cua-id-stop) Stop a CUA task\n\nput\n\nhttps://app.hyperbrowser.ai/api/task/cua/{id}/stop\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nCUA task stopped successfully\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request PUT \\\n --url 'https://app.hyperbrowser.ai/api/task/cua/{id}/stop' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"success\": true\n}\n```\n\nCUA task stopped successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua\\#api-task-cua-id-status) Get CUA task status\n\nget\n\nhttps://app.hyperbrowser.ai/api/task/cua/{id}/status\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nCUA task status\n\napplication/json\n\n404\n\nCUA task not found\n\napplication/json\n\n500\n\nServer error\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/task/cua/{id}/status' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\n500\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"status\": \"pending\"\n}\n```\n\nCUA task status\n\n[PreviousClaude Computer Use](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/claude-computer-use) [NextProfiles](https://www.docs.hyperbrowser.ai/reference/api-reference/profiles)\n\nLast updated 16 hours ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua", "title": "OpenAI CUA | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/FLJPkGpsU8OBGbfnwDTT", "og:title": "OpenAI CUA | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/FLJPkGpsU8OBGbfnwDTT", "twitter:title": "OpenAI CUA | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua", "status": "completed", "error": null }, "summary": "OpenAI CUA for Hyperbrowser details integration and automation features for effective web scraping." }, { "pathname": "/reference/api-reference/profiles", "data": { "markdown": "Profiles \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/profiles\\#api-profile) Creates a new profile\n\npost\n\nhttps://app.hyperbrowser.ai/api/profile\n\nAuthorizations\n\nResponses\n\n200\n\nProfile created\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request POST \\\n --url 'https://app.hyperbrowser.ai/api/profile' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"id\": \"text\"\n}\n```\n\nProfile created\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/profiles\\#api-profile-id) Get profile by ID\n\nget\n\nhttps://app.hyperbrowser.ai/api/profile/{id}\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nProfile details\n\napplication/json\n\n404\n\nProfile not found\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/profile/{id}' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\n404\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"id\": \"text\",\n \"teamId\": \"text\",\n \"createdAt\": \"text\",\n \"updatedAt\": \"text\"\n}\n```\n\nProfile details\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/profiles\\#api-profile-id-1) Delete profile by ID\n\ndelete\n\nhttps://app.hyperbrowser.ai/api/profile/{id}\n\nAuthorizations\n\nPath parameters\n\nidstringrequired\n\nResponses\n\n200\n\nProfile deleted\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request DELETE \\\n --url 'https://app.hyperbrowser.ai/api/profile/{id}' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"success\": true\n}\n```\n\nProfile deleted\n\n[PreviousOpenAI CUA](https://www.docs.hyperbrowser.ai/reference/api-reference/agents/openai-cua) [NextExtensions](https://www.docs.hyperbrowser.ai/reference/api-reference/extensions)\n\nLast updated 2 months ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference/profiles", "title": "Profiles | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/jB92biHeDLo342uyhWp5", "og:title": "Profiles | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference/profiles", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/jB92biHeDLo342uyhWp5", "twitter:title": "Profiles | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference/profiles", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference/profiles", "status": "completed", "error": null }, "summary": "The \"Profiles\" page in Hyperbrowser documentation explains how to manage and configure user profiles for scraping tasks." }, { "pathname": "/reference/api-reference/extensions", "data": { "markdown": "Extensions \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/extensions\\#api-extensions-add) Add a new extension\n\npost\n\nhttps://app.hyperbrowser.ai/api/extensions/add\n\nAuthorizations\n\nBody\n\nfilestring 路 binaryrequired\n\nnamestring\n\nResponses\n\n200\n\nExtension added successfully\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --request POST \\\n --url 'https://app.hyperbrowser.ai/api/extensions/add' \\\n --header 'x-api-key: YOUR_API_KEY' \\\n --header 'Content-Type: multipart/form-data' \\\n --form 'file=binary' \\\n --form 'name=text'\n```\n\nTest it\n\n200\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n{\n \"id\": \"text\",\n \"name\": \"text\",\n \"createdAt\": \"text\",\n \"updatedAt\": \"text\"\n}\n```\n\nExtension added successfully\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/reference/api-reference/extensions\\#api-extensions-list) List all extensions\n\nget\n\nhttps://app.hyperbrowser.ai/api/extensions/list\n\nAuthorizations\n\nResponses\n\n200\n\nExtension added successfully\n\napplication/json\n\ncURL\n\nJavaScript\n\nPython\n\nHTTP\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ncurl -L \\\n --url 'https://app.hyperbrowser.ai/api/extensions/list' \\\n --header 'x-api-key: YOUR_API_KEY'\n```\n\nTest it\n\n200\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\n[\\\n {\\\n \"id\": \"text\",\\\n \"name\": \"text\",\\\n \"createdAt\": \"text\",\\\n \"updatedAt\": \"text\"\\\n }\\\n]\n```\n\nExtension added successfully\n\n[PreviousProfiles](https://www.docs.hyperbrowser.ai/reference/api-reference/profiles) [NextLangChain](https://www.docs.hyperbrowser.ai/integrations/langchain)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/reference/api-reference/extensions", "title": "Extensions | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/QhdXTG0tAZ8Gs1Kgu9ta", "og:title": "Extensions | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/reference/api-reference/extensions", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/QhdXTG0tAZ8Gs1Kgu9ta", "twitter:title": "Extensions | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/reference/api-reference/extensions", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/reference/api-reference/extensions", "status": "completed", "error": null }, "summary": "The page discusses extensions for Hyperbrowser that enhance its web scraping and automation capabilities." }, { "pathname": "/integrations/langchain", "data": { "markdown": "LangChain \\| Hyperbrowser\n\nHyperbrowser provides a Document Loader integration with LangChain via the `langchain-hyperbrowser` package. It can be used to load the metadata and contents(in formatted markdown or html) of any site as a LangChain `Document`.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/integrations/langchain\\#installation-and-setup) Installation and Setup\n\nTo get started with `langchain-hyperbrowser`, you can install the package using pip:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install langchain-hyperbrowser\n```\n\nAnd you should configure credentials by setting the following environment variables:\n\n`HYPERBROWSER_API_KEY=`\n\nYou can get an API Key easily from the [dashboard](https://app.hyperbrowser.ai/). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY` or you can pass it via the `api_key` argument in the constructor.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/integrations/langchain\\#document-loader) Document Loader\n\nThe `HyperbrowserLoader` class in `langchain-hyperbrowser` can easily be used to load content from any single page or multiple pages as well as crawl an entire site. The content can be loaded as markdown or html.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nfrom langchain_hyperbrowser import HyperbrowserLoader\n\nloader = HyperbrowserLoader(urls=\"https://example.com\")\ndocs = loader.load()\n\nprint(docs[0])\n```\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/integrations/langchain\\#advanced-usage) Advanced Usage\n\nYou can specify the operation to be performed by the loader. The default operation is `scrape`. For `scrape`, you can provide a single URL or a list of URLs to be scraped. For `crawl`, you can only provide a single URL. The `crawl` operation will crawl the provided page and subpages and return a document for each page.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nloader = HyperbrowserLoader(\n urls=\"https://hyperbrowser.ai\", api_key=\"YOUR_API_KEY\", operation=\"crawl\"\n)\n```\n\nOptional params for the loader can also be provided in the `params` argument. For more information on the supported params, you can see the params for [scraping](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape#start-scrape-job) or [crawling](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl#start-scrape-job).\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nloader = HyperbrowserLoader(\n urls=\"https://example.com\",\n api_key=\"YOUR_API_KEY\",\n operation=\"scrape\",\n params={\"scrape_options\": {\"include_tags\": [\"h1\", \"h2\", \"p\"]}}\n)\n```\n\n[PreviousExtensions](https://www.docs.hyperbrowser.ai/reference/api-reference/extensions) [NextLlamaIndex](https://www.docs.hyperbrowser.ai/integrations/llamaindex)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/integrations/langchain", "title": "LangChain | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/1catOGaWWrhyySbb6Ddw", "og:title": "LangChain | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/integrations/langchain", "description": "Using Hyperbrowser's Document Loader Integration", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/1catOGaWWrhyySbb6Ddw", "twitter:title": "LangChain | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/integrations/langchain", "og:description": "Using Hyperbrowser's Document Loader Integration", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Using Hyperbrowser's Document Loader Integration" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/integrations/langchain", "status": "completed", "error": null }, "summary": "LangChain integrates with Hyperbrowser for enhanced web scraping and automation capabilities." }, { "pathname": "/integrations/llamaindex", "data": { "markdown": "LlamaIndex \\| Hyperbrowser\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/integrations/llamaindex\\#installation-and-setup) Installation and Setup\n\nTo get started with LlamaIndex and Hyperbrowser, you can install the necessary packages using pip:\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\npip install llama-index-core llama-index-readers-web hyperbrowser\n```\n\nAnd you should configure credentials by setting the following environment variables:\n\n`HYPERBROWSER_API_KEY=`\n\nYou can get an API Key easily from the [dashboard](https://app.hyperbrowser.ai/). Once you have your API Key, add it to your `.env` file as `HYPERBROWSER_API_KEY` or you can pass it via the `api_key` argument in the `HyperbrowserWebReader` constructor.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/integrations/llamaindex\\#usage) Usage\n\nOnce you have your API Key and have installed the packages you can load webpages into LlamaIndex using `HyperbrowserWebReader`.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nfrom llama_index.readers.web import HyperbrowserWebReader\n\nreader = HyperbrowserWebReader(api_key=\"your_api_key_here\")\n```\n\nTo load data, you can specify the operation to be performed by the loader. The default operation is `scrape`. For `scrape`, you can provide a single URL or a list of URLs to be scraped. For `crawl`, you can only provide a single URL. The `crawl` operation will crawl the provided page and subpages and return a document for each page. `HyperbrowserWebReader` supports loading and lazy loading data in both sync and async modes.\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ndocuments = reader.load_data(\n urls=[\"https://example.com\"],\n operation=\"scrape\",\n)\n```\n\nOptional params for the loader can also be provided in the `params` argument. For more information on the supported params, you can see the params for [scraping](https://www.docs.hyperbrowser.ai/reference/sdks/python/scrape#start-scrape-job) or [crawling](https://www.docs.hyperbrowser.ai/reference/sdks/python/crawl#start-scrape-job).\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\ndocuments = reader.load_data(\n urls=[\"https://example.com\"],\n operation=\"scrape\",\n params={\"scrape_options\": {\"include_tags\": [\"h1\", \"h2\", \"p\"]}},\n)\n```\n\n[PreviousLangChain](https://www.docs.hyperbrowser.ai/integrations/langchain)\n\nLast updated 1 month ago", "metadata": { "url": "https://docs.hyperbrowser.ai/integrations/llamaindex", "title": "LlamaIndex | Hyperbrowser", "robots": "index, follow", "og:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/dLXC9OE55JHCKMKtiQ0K", "og:title": "LlamaIndex | Hyperbrowser", "viewport": "width=device-width, initial-scale=1", "generator": "GitBook (fc00b51)", "link:icon": "https://docs.hyperbrowser.ai/~gitbook/image?url=https%3A%2F%2F4086371784-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Forganizations%252F-MjSP1cG6WWBpRLZETsb%252Fsites%252Fsite_47QD9%252Ficon%252FdJ3PfxcwhL57mzu3TQ7n%252Fhb-svg%25201%2520%281%29.png%3Falt%3Dmedia%26token%3D73062494-e327-43d6-bb23-c3511710e2aa&width=48&height=48&sign=71de792b&sv=2", "sourceURL": "https://www.docs.hyperbrowser.ai/integrations/llamaindex", "description": "Using Hyperbrowser's Web Reader Integration", "color-scheme": "dark", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "twitter:card": "summary_large_image", "twitter:image": "https://docs.hyperbrowser.ai/~gitbook/ogimage/dLXC9OE55JHCKMKtiQ0K", "twitter:title": "LlamaIndex | Hyperbrowser", "link:canonical": "https://docs.hyperbrowser.ai/integrations/llamaindex", "og:description": "Using Hyperbrowser's Web Reader Integration", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css", "twitter:description": "Using Hyperbrowser's Web Reader Integration" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/integrations/llamaindex", "status": "completed", "error": null }, "summary": "LlamaIndex documentation for integrating with Hyperbrowser for web scraping and automation tasks." }, { "pathname": "/~gitbook/pdf", "data": { "markdown": "Hyperbrowser\n\n[Go back to content](https://docs.hyperbrowser.ai/ \"Go back to content\")\n\n[All pages](https://www.docs.hyperbrowser.ai/~gitbook/pdf?limit=100)\n\n[Powered by GitBook](https://www.gitbook.com/?utm_source=content&utm_medium=trademark&utm_campaign=fwZVRs9Jmwzw9cfbchYG)\n\n1 of 1\n\n# Welcome to Hyperbrowser\n\nWelcome to Hyperbrowser, the Internet for AI. Hyperbrowser is the next-generation platform empowering AI agents and enabling effortless, scalable browser automation. Built specifically for AI developers, it eliminates the headaches of local infrastructure and performance bottlenecks, allowing you to focus entirely on building your solutions, rather getting gummed up on browser problems.\n\nWhether you're training AI agents to navigate the web, collecting data for model fine-tuning, testing applications, or simply scraping data, Hyperbrowser lets you launch and manage browser sessions with ease鈥攏o complicated setup required. Our platform provides streamlined solutions for all your web scraping needs, from single-page extraction to comprehensive site crawling.\n\n### [Direct link to heading](https://www.docs.hyperbrowser.ai/~gitbook/pdf?page=kzTlst3tKo255yz4YpDi&only=yes&limit=100\\#pdf-page-kzTlst3tKo255yz4YpDi-why-developers-choose-hyperbrowser) Why Developers Choose Hyperbrowser:\n\n- **Instant Scalability** \\- Deploy hundreds of AI agent browser sessions in seconds without infrastructure complexity\n\n- **Powerful APIs** \\- Purpose-built APIs for managing sessions, training environments, scraping/crawling sites, and enhancing AI capabilities\n\n- **Production-Ready AI Infrastructure** \\- Enterprise-grade reliability and security built specifically for AI workloads\n\n- **Advanced Anti-Bot Protection Bypass** \\- Built-in stealth mode, ad blocking, automatic CAPTCHA solving, and rotating proxies for uninterrupted AI operation\n\n- **AI-First Design** \\- Native support for multiple AI frameworks including LangChain, LlamaIndex, MCP, and more\n\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/~gitbook/pdf?page=kzTlst3tKo255yz4YpDi&only=yes&limit=100\\#pdf-page-kzTlst3tKo255yz4YpDi-quick-example) Quick Example\n\nStart automating in just a few lines of code\n\nCopy\n\n```inline-grid min-w-full grid-cols-[auto_1fr] p-2 [count-reset:line]\nimport { Hyperbrowser } from \"@hyperbrowser/sdk\";\nimport { connect } from \"puppeteer-core\";\n\nconst client = new Hyperbrowser({\n apiKey: process.env.HYPERBROWSER_API_KEY,\n});\n\nconst session = await client.sessions.create();\n\n// Use the session to automate browser actions\nconst browser = await connect({\n browserWSEndpoint: session.wsEndpoint,\n defaultViewport: null,\n});\n\n// Use the browser to automate browser actions\nconst page = await browser.newPage();\nawait page.goto(\"https://example.com\");\n\nawait browser.close();\n\n// Once done, you can stop the session\nawait client.sessions.stop(session.id);\n```\n\n## [Direct link to heading](https://www.docs.hyperbrowser.ai/~gitbook/pdf?page=kzTlst3tKo255yz4YpDi&only=yes&limit=100\\#pdf-page-kzTlst3tKo255yz4YpDi-jump-right-in) Jump right in\n\n[馃摉\\\\\n\\\\\n**Scraping**\\\\\n\\\\\nScrape a site and get its contents in markdown](https://app.gitbook.com/s/fwZVRs9Jmwzw9cfbchYG/get-started/quickstart/scraping) [馃懆鈥嶐煉籠\\\\n\\\\\n**Puppeteer**\\\\\n\\\\\nConnect to a browser session with Puppeteer](https://app.gitbook.com/s/fwZVRs9Jmwzw9cfbchYG/get-started/quickstart/puppeteer) [馃暦锔廫\\\\n\\\\\n**Crawling**\\\\\n\\\\\nCrawl an entire site and all its linked pages](https://app.gitbook.com/s/fwZVRs9Jmwzw9cfbchYG/get-started/quickstart/crawling) [馃椉\\\\\n\\\\\nStructured Extraction\\\\\n\\\\\nPerform extraction into a json schema](https://app.gitbook.com/s/fwZVRs9Jmwzw9cfbchYG/web-scraping/extract)", "metadata": { "url": "https://docs.hyperbrowser.ai/~gitbook/pdf?page=kzTlst3tKo255yz4YpDi&only=yes&limit=100", "title": "Hyperbrowser", "robots": "noindex, nofollow", "viewport": "width=device-width, initial-scale=1", "sourceURL": "https://www.docs.hyperbrowser.ai/~gitbook/pdf?page=kzTlst3tKo255yz4YpDi&only=yes&limit=100", "link:preload": "https://static.gitbook.com/_next/static/chunks/webpack-3669b6c2f2acdd0a.js", "link:preconnect": "https://static.gitbook.com/", "link:stylesheet": "https://static.gitbook.com/_next/static/css/4af9aafd612346fe.css" }, "html": null, "url": "https://www.docs.hyperbrowser.ai/~gitbook/pdf?page=kzTlst3tKo255yz4YpDi&only=yes&limit=100", "status": "completed", "error": null }, "summary": "Hyperbrowser is a web scraping and automation tool, offering extensive documentation for users." } ]