# Directory Structure
```
├── .gitignore
├── cline_mcp_settings.json.example
├── CONTRIBUTING.md
├── examples
│ ├── basic-worker-example.js
│ ├── debugging-tools
│ │ └── debug-test.js
│ ├── minimal-worker-example.js
│ └── testing
│ └── content-test.js
├── experiments
│ ├── basic-rest-api
│ │ └── index.ts
│ ├── content-extraction
│ │ └── index.ts
│ └── puppeteer-binding
│ └── index.ts
├── LICENSE
├── package-lock.json
├── package.json
├── puppeteer-worker.js
├── README.md
├── src
│ ├── browser-client.ts
│ ├── content-processor.ts
│ ├── index.ts
│ └── server.ts
├── test-puppeteer.js
├── tsconfig.json
└── wrangler.toml
```
# Files
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
```
# Node.js dependencies
node_modules/
# Build output
dist/
# Environment variables
.env
.env.local
.env.development.local
.env.test.local
.env.production.local
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# Editor directories and files
.idea/
.vscode/
*.swp
*.swo
.DS_Store
```
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
```markdown
# Cloudflare Browser Rendering Experiments & MCP Server
This project demonstrates how to use Cloudflare Browser Rendering to extract web content for LLM context. It includes experiments with the REST API and Workers Binding API, as well as an MCP server implementation that can be used to provide web context to LLMs.
<a href="https://glama.ai/mcp/servers/wg9fikq571">
<img width="380" height="200" src="https://glama.ai/mcp/servers/wg9fikq571/badge" alt="Web Content Server MCP server" />
</a>
## Project Structure
```
cloudflare-browser-rendering/
├── examples/ # Example implementations and utilities
│ ├── basic-worker-example.js # Basic Worker with Browser Rendering
│ ├── minimal-worker-example.js # Minimal implementation
│ ├── debugging-tools/ # Tools for debugging
│ │ └── debug-test.js # Debug test utility
│ └── testing/ # Testing utilities
│ └── content-test.js # Content testing utility
├── experiments/ # Educational experiments
│ ├── basic-rest-api/ # REST API tests
│ ├── puppeteer-binding/ # Workers Binding API tests
│ └── content-extraction/ # Content processing tests
├── src/ # MCP server source code
│ ├── index.ts # Main entry point
│ ├── server.ts # MCP server implementation
│ ├── browser-client.ts # Browser Rendering client
│ └── content-processor.ts # Content processing utilities
├── puppeteer-worker.js # Cloudflare Worker with Browser Rendering binding
├── test-puppeteer.js # Tests for the main implementation
├── wrangler.toml # Wrangler configuration for the Worker
├── cline_mcp_settings.json.example # Example MCP settings for Cline
├── .gitignore # Git ignore file
└── LICENSE # MIT License
```
## Prerequisites
- Node.js (v16 or later)
- A Cloudflare account with Browser Rendering enabled
- TypeScript
- Wrangler CLI (for deploying the Worker)
## Installation
1. Clone the repository:
```bash
git clone https://github.com/yourusername/cloudflare-browser-rendering.git
cd cloudflare-browser-rendering
```
2. Install dependencies:
```bash
npm install
```
## Cloudflare Worker Setup
1. Install the Cloudflare Puppeteer package:
```bash
npm install @cloudflare/puppeteer
```
2. Configure Wrangler:
```toml
# wrangler.toml
name = "browser-rendering-api"
main = "puppeteer-worker.js"
compatibility_date = "2023-10-30"
compatibility_flags = ["nodejs_compat"]
[browser]
binding = "browser"
```
3. Deploy the Worker:
```bash
npx wrangler deploy
```
4. Test the Worker:
```bash
node test-puppeteer.js
```
## Running the Experiments
### Basic REST API Experiment
This experiment demonstrates how to use the Cloudflare Browser Rendering REST API to fetch and process web content:
```bash
npm run experiment:rest
```
### Puppeteer Binding API Experiment
This experiment demonstrates how to use the Cloudflare Browser Rendering Workers Binding API with Puppeteer for more advanced browser automation:
```bash
npm run experiment:puppeteer
```
### Content Extraction Experiment
This experiment demonstrates how to extract and process web content specifically for use as context in LLMs:
```bash
npm run experiment:content
```
## MCP Server
The MCP server provides tools for fetching and processing web content using Cloudflare Browser Rendering for use as context in LLMs.
### Building the MCP Server
```bash
npm run build
```
### Running the MCP Server
```bash
npm start
```
Or, for development:
```bash
npm run dev
```
### MCP Server Tools
The MCP server provides the following tools:
1. `fetch_page` - Fetches and processes a web page for LLM context
2. `search_documentation` - Searches Cloudflare documentation and returns relevant content
3. `extract_structured_content` - Extracts structured content from a web page using CSS selectors
4. `summarize_content` - Summarizes web content for more concise LLM context
## Configuration
To use your Cloudflare Browser Rendering endpoint, set the `BROWSER_RENDERING_API` environment variable:
```bash
export BROWSER_RENDERING_API=https://YOUR_WORKER_URL_HERE
```
Replace `YOUR_WORKER_URL_HERE` with the URL of your deployed Cloudflare Worker. You'll need to replace this placeholder in several files:
1. In test files: `test-puppeteer.js`, `examples/debugging-tools/debug-test.js`, `examples/testing/content-test.js`
2. In the MCP server configuration: `cline_mcp_settings.json.example`
3. In the browser client: `src/browser-client.ts` (as a fallback if the environment variable is not set)
## Integrating with Cline
To integrate the MCP server with Cline, copy the `cline_mcp_settings.json.example` file to the appropriate location:
```bash
cp cline_mcp_settings.json.example ~/Library/Application\ Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json
```
Or add the configuration to your existing `cline_mcp_settings.json` file.
## Key Learnings
1. Cloudflare Browser Rendering requires the `@cloudflare/puppeteer` package to interact with the browser binding.
2. The correct pattern for using the browser binding is:
```javascript
import puppeteer from '@cloudflare/puppeteer';
// Then in your handler:
const browser = await puppeteer.launch(env.browser);
const page = await browser.newPage();
```
3. When deploying a Worker that uses the Browser Rendering binding, you need to enable the `nodejs_compat` compatibility flag.
4. Always close the browser after use to avoid resource leaks.
## License
MIT
```
--------------------------------------------------------------------------------
/CONTRIBUTING.md:
--------------------------------------------------------------------------------
```markdown
# Contributing to Cloudflare Browser Rendering MCP Server
Thank you for your interest in contributing to this project! This document provides guidelines and instructions for contributing.
## Code of Conduct
Please be respectful and considerate of others when contributing to this project.
## How to Contribute
1. Fork the repository
2. Create a new branch for your feature or bug fix
3. Make your changes
4. Run tests to ensure your changes don't break existing functionality
5. Submit a pull request
## Development Setup
1. Clone the repository:
```bash
git clone https://github.com/yourusername/cloudflare-browser-rendering.git
cd cloudflare-browser-rendering
```
2. Install dependencies:
```bash
npm install
```
3. Build the project:
```bash
npm run build
```
## Project Structure
Please refer to the README.md for a detailed explanation of the project structure.
## Testing
Before submitting a pull request, please ensure that your changes pass all tests:
```bash
npm test
```
## Pull Request Process
1. Update the README.md with details of changes if applicable
2. Update the examples if you've added new functionality
3. The PR will be merged once it's reviewed and approved
## Adding New Features
If you're adding a new feature:
1. Add appropriate tests
2. Update documentation
3. Add examples if applicable
## Reporting Bugs
When reporting bugs, please include:
1. A clear description of the bug
2. Steps to reproduce
3. Expected behavior
4. Actual behavior
5. Environment details (OS, Node.js version, etc.)
## Feature Requests
Feature requests are welcome. Please provide a clear description of the feature and why it would be beneficial to the project.
## License
By contributing to this project, you agree that your contributions will be licensed under the project's MIT License.
```
--------------------------------------------------------------------------------
/wrangler.toml:
--------------------------------------------------------------------------------
```toml
# Cloudflare Worker with Browser Rendering binding
name = "browser-rendering-api"
main = "puppeteer-worker.js"
compatibility_date = "2023-10-30"
compatibility_flags = ["nodejs_compat"]
# Configure the Browser Rendering binding
[browser]
binding = "browser"
```
--------------------------------------------------------------------------------
/tsconfig.json:
--------------------------------------------------------------------------------
```json
{
"compilerOptions": {
"target": "ES2020",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"esModuleInterop": true,
"strict": true,
"outDir": "dist",
"declaration": true,
"sourceMap": true,
"resolveJsonModule": true
},
"include": ["src/**/*", "experiments/**/*"],
"exclude": ["node_modules", "dist"]
}
```
--------------------------------------------------------------------------------
/src/index.ts:
--------------------------------------------------------------------------------
```typescript
#!/usr/bin/env node
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
import { BrowserRenderingServer } from './server.js';
/**
* Main entry point for the Cloudflare Browser Rendering MCP server
*/
async function main() {
try {
const server = new BrowserRenderingServer();
await server.run();
console.error('Cloudflare Browser Rendering MCP server running on stdio');
} catch (error) {
console.error('Failed to start MCP server:', error);
process.exit(1);
}
}
// Run the server
main().catch(console.error);
```
--------------------------------------------------------------------------------
/examples/minimal-worker-example.js:
--------------------------------------------------------------------------------
```javascript
/**
* Minimal Cloudflare Worker with Browser Rendering binding
*
* This is a minimal example that just returns information about
* the environment and the browser binding.
*/
export default {
async fetch(request, env, ctx) {
// Check if browser binding exists
const hasBrowser = 'browser' in env;
// Return information about the environment
return new Response(JSON.stringify({
env: Object.keys(env),
hasBrowser,
browser: hasBrowser ? {
type: typeof env.browser,
methods: Object.getOwnPropertyNames(Object.getPrototypeOf(env.browser) || {})
.filter(prop => typeof env.browser[prop] === 'function'),
} : null,
}, null, 2), {
headers: { 'Content-Type': 'application/json' },
});
},
};
```
--------------------------------------------------------------------------------
/examples/basic-worker-example.js:
--------------------------------------------------------------------------------
```javascript
/**
* Basic Cloudflare Worker with Browser Rendering binding
*
* This is a simple example of how to use the Browser Rendering binding
* in a Cloudflare Worker.
*/
export default {
async fetch(request, env, ctx) {
// Get the URL from the request query parameters
const url = new URL(request.url);
const targetUrl = url.searchParams.get('url') || 'https://example.com';
// Create a response with information about the request
const info = {
url: targetUrl,
timestamp: new Date().toISOString(),
worker: {
name: 'browser-rendering-example',
environment: Object.keys(env),
},
};
// Return the information as JSON
return new Response(JSON.stringify(info, null, 2), {
headers: { 'Content-Type': 'application/json' },
});
},
};
```
--------------------------------------------------------------------------------
/package.json:
--------------------------------------------------------------------------------
```json
{
"name": "cloudflare-browser-rendering",
"version": "1.0.0",
"description": "MCP server for providing web context to LLMs using Cloudflare Browser Rendering",
"main": "dist/src/index.js",
"scripts": {
"test": "node test-puppeteer.js",
"build": "tsc",
"start": "node dist/src/index.js",
"dev": "ts-node src/index.ts",
"experiment:rest": "ts-node experiments/basic-rest-api/index.ts",
"experiment:puppeteer": "ts-node experiments/puppeteer-binding/index.ts",
"experiment:content": "ts-node experiments/content-extraction/index.ts",
"deploy:worker": "npx wrangler deploy"
},
"keywords": [
"cloudflare",
"browser-rendering",
"mcp",
"llm",
"context"
],
"author": "",
"license": "MIT",
"devDependencies": {
"@types/node": "^22.13.5",
"ts-node": "^10.9.2",
"typescript": "^5.7.3",
"wrangler": "^3.111.0"
},
"dependencies": {
"@cloudflare/puppeteer": "^0.0.14",
"@modelcontextprotocol/sdk": "^1.6.1",
"axios": "^1.8.1"
}
}
```
--------------------------------------------------------------------------------
/examples/debugging-tools/debug-test.js:
--------------------------------------------------------------------------------
```javascript
/**
* Debug Test for Cloudflare Browser Rendering
*
* This script helps debug issues with the Browser Rendering binding
* by making a request to the info endpoint of the Worker.
*/
const axios = require('axios');
// The URL of the deployed Worker
// Replace YOUR_WORKER_URL_HERE with your actual worker URL when testing
const WORKER_URL = 'https://YOUR_WORKER_URL_HERE';
/**
* Test the info endpoint
*/
async function testInfoEndpoint() {
console.log('Testing info endpoint...');
try {
// Make a request to the info endpoint
const response = await axios.get(`${WORKER_URL}/info`);
console.log('Response status:', response.status);
console.log('Response data:', response.data);
return response.data;
} catch (error) {
console.error('Error testing info endpoint:', error.message);
if (error.response) {
console.error('Response data:', error.response.data);
console.error('Response status:', error.response.status);
}
throw error;
}
}
/**
* Main function to run the tests
*/
async function runTests() {
try {
// Test the info endpoint
await testInfoEndpoint();
console.log('\nTests completed successfully!');
} catch (error) {
console.error('Tests failed:', error);
}
}
// Run the tests
runTests();
```
--------------------------------------------------------------------------------
/examples/testing/content-test.js:
--------------------------------------------------------------------------------
```javascript
/**
* Content Test for Cloudflare Browser Rendering
*
* This script tests the content fetching functionality of the Worker
* by making a request to the content endpoint.
*/
const axios = require('axios');
// The URL of the deployed Worker
// Replace YOUR_WORKER_URL_HERE with your actual worker URL when testing
const WORKER_URL = 'https://YOUR_WORKER_URL_HERE';
/**
* Test the content endpoint
*/
async function testContentEndpoint() {
console.log('Testing content endpoint...');
try {
// Make a request to the content endpoint
const response = await axios.post(`${WORKER_URL}/content`, {
url: 'https://example.com',
rejectResourceTypes: ['image', 'font', 'media'],
});
console.log('Response status:', response.status);
console.log('Content length:', response.data.content.length);
console.log('Content preview:', response.data.content.substring(0, 200) + '...');
return response.data.content;
} catch (error) {
console.error('Error testing content endpoint:', error.message);
if (error.response) {
console.error('Response data:', error.response.data);
}
throw error;
}
}
/**
* Main function to run the tests
*/
async function runTests() {
try {
// Test the content endpoint
await testContentEndpoint();
console.log('\nTests completed successfully!');
} catch (error) {
console.error('Tests failed:', error);
}
}
// Run the tests
runTests();
```
--------------------------------------------------------------------------------
/src/browser-client.ts:
--------------------------------------------------------------------------------
```typescript
import axios from 'axios';
/**
* Client for interacting with Cloudflare Browser Rendering
*/
export class BrowserClient {
private apiEndpoint: string;
constructor() {
// Use the deployed Cloudflare Worker or a default placeholder
// Replace YOUR_WORKER_URL_HERE with your actual worker URL when deploying
this.apiEndpoint = process.env.BROWSER_RENDERING_API || 'https://YOUR_WORKER_URL_HERE';
}
/**
* Fetches rendered HTML content from a URL
* @param url The URL to fetch content from
* @returns The rendered HTML content
*/
async fetchContent(url: string): Promise<string> {
try {
console.log(`Fetching content from: ${url}`);
// Make the API call to the Cloudflare Worker
const response = await axios.post(`${this.apiEndpoint}/content`, {
url,
rejectResourceTypes: ['image', 'font', 'media'],
waitUntil: 'networkidle0',
});
return response.data.content;
} catch (error) {
console.error('Error fetching content:', error);
throw new Error(`Failed to fetch content: ${error instanceof Error ? error.message : String(error)}`);
}
}
/**
* Takes a screenshot of a URL
* @param url The URL to take a screenshot of
* @returns Base64-encoded screenshot image
*/
async takeScreenshot(url: string): Promise<string> {
try {
console.log(`Taking screenshot of: ${url}`);
// Make the API call to the Cloudflare Worker
const response = await axios.post(`${this.apiEndpoint}/screenshot`, {
url,
fullPage: false,
width: 1280,
height: 800,
});
return response.data.screenshot;
} catch (error) {
console.error('Error taking screenshot:', error);
throw new Error(`Failed to take screenshot: ${error instanceof Error ? error.message : String(error)}`);
}
}
}
```
--------------------------------------------------------------------------------
/test-puppeteer.js:
--------------------------------------------------------------------------------
```javascript
const axios = require('axios');
// The URL of the deployed Worker
// Replace YOUR_WORKER_URL_HERE with your actual worker URL when testing
const WORKER_URL = 'https://YOUR_WORKER_URL_HERE';
// Test the /content endpoint
async function testContentEndpoint() {
console.log('Testing /content endpoint...');
try {
const response = await axios.post(`${WORKER_URL}/content`, {
url: 'https://example.com',
rejectResourceTypes: ['image', 'font', 'media'],
});
console.log('Response status:', response.status);
console.log('Content length:', response.data.content.length);
console.log('Content preview:', response.data.content.substring(0, 200) + '...');
return response.data.content;
} catch (error) {
console.error('Error testing /content endpoint:', error.message);
if (error.response) {
console.error('Response data:', error.response.data);
}
throw error;
}
}
// Test the /screenshot endpoint
async function testScreenshotEndpoint() {
console.log('\nTesting /screenshot endpoint...');
try {
const response = await axios.post(`${WORKER_URL}/screenshot`, {
url: 'https://example.com',
width: 800,
height: 600,
});
console.log('Response status:', response.status);
console.log('Screenshot data length:', response.data.screenshot.length);
console.log('Screenshot data preview:', response.data.screenshot.substring(0, 50) + '...');
return response.data.screenshot;
} catch (error) {
console.error('Error testing /screenshot endpoint:', error.message);
if (error.response) {
console.error('Response data:', error.response.data);
}
throw error;
}
}
// Main function to run the tests
async function runTests() {
try {
// Test the /content endpoint
await testContentEndpoint();
// Test the /screenshot endpoint
await testScreenshotEndpoint();
console.log('\nTests completed successfully!');
} catch (error) {
console.error('Tests failed:', error);
}
}
// Run the tests
runTests();
```
--------------------------------------------------------------------------------
/experiments/basic-rest-api/index.ts:
--------------------------------------------------------------------------------
```typescript
import axios from 'axios';
// Cloudflare Browser Rendering REST API endpoint
// Note: In a real implementation, this would be your Cloudflare account's endpoint
const BROWSER_RENDERING_API = 'https://browser-rendering.youraccount.workers.dev';
/**
* Fetches rendered HTML content from a URL using Cloudflare Browser Rendering
* @param url The URL to fetch content from
* @returns The rendered HTML content
*/
async function fetchRenderedContent(url: string): Promise<string> {
try {
const response = await axios.post(`${BROWSER_RENDERING_API}/content`, {
url,
// Optional parameters to optimize performance
rejectResourceTypes: ['image', 'font', 'media'],
// Wait for network to be idle (no requests for 500ms)
waitUntil: 'networkidle0',
});
return response.data.content;
} catch (error) {
if (axios.isAxiosError(error)) {
console.error('Error fetching content:', error.message);
if (error.response) {
console.error('Response data:', error.response.data);
}
} else {
console.error('Unexpected error:', error);
}
throw error;
}
}
/**
* Extracts main content from Cloudflare documentation HTML
* @param html The full HTML content
* @returns The extracted main content
*/
function extractMainContent(html: string): string {
// In a real implementation, we would use a proper HTML parser
// For this experiment, we'll use a simple regex approach
// Try to find the main content container in Cloudflare docs
const mainContentMatch = html.match(/<main[^>]*>([\s\S]*?)<\/main>/i);
if (mainContentMatch && mainContentMatch[1]) {
return mainContentMatch[1];
}
return 'Could not extract main content';
}
/**
* Main function to run the experiment
*/
async function runExperiment() {
console.log('Starting Browser Rendering REST API experiment...');
// Fetch content from Cloudflare docs
const url = 'https://developers.cloudflare.com/browser-rendering/';
console.log(`Fetching content from: ${url}`);
try {
// Note: In a real implementation, you would use your actual Cloudflare Browser Rendering endpoint
// For this experiment, we'll simulate the response
console.log('This is a simulation since we need a real Cloudflare Browser Rendering endpoint');
// Simulated content
const simulatedHtml = `
<!DOCTYPE html>
<html>
<head>
<title>Cloudflare Browser Rendering</title>
</head>
<body>
<header>
<nav>Navigation</nav>
</header>
<main>
<h1>Browser Rendering</h1>
<p>Cloudflare Browser Rendering is a serverless headless browser service that allows execution of browser actions within Cloudflare Workers.</p>
<h2>Features</h2>
<ul>
<li>REST API for simple operations</li>
<li>Workers Binding API for complex automation</li>
<li>Puppeteer integration</li>
</ul>
</main>
<footer>Footer content</footer>
</body>
</html>
`;
// Extract main content
const mainContent = extractMainContent(simulatedHtml);
console.log('\nExtracted main content:');
console.log(mainContent);
console.log('\nIn a real implementation, you would:');
console.log('1. Replace the BROWSER_RENDERING_API constant with your actual endpoint');
console.log('2. Uncomment the fetchRenderedContent call');
console.log('3. Use a proper HTML parser for content extraction');
} catch (error) {
console.error('Experiment failed:', error);
}
}
// Run the experiment
runExperiment();
```
--------------------------------------------------------------------------------
/src/content-processor.ts:
--------------------------------------------------------------------------------
```typescript
/**
* Processes web content for LLM context
*/
export class ContentProcessor {
/**
* Processes HTML content for LLM context
* @param html The HTML content to process
* @param url The URL of the content
* @returns Processed content suitable for LLM context
*/
processForLLM(html: string, url: string): string {
// Extract metadata
const metadata = this.extractMetadata(html, url);
// Clean the content
const cleanedContent = this.cleanContent(html);
// Format for LLM context
return this.formatForLLM(cleanedContent, metadata);
}
/**
* Extracts metadata from HTML content
* @param html The HTML content
* @param url The URL of the content
* @returns Metadata object
*/
private extractMetadata(html: string, url: string): Record<string, string> {
const titleMatch = html.match(/<title[^>]*>([\s\S]*?)<\/title>/i);
const descriptionMatch = html.match(/<meta name="description" content="([^"]*)">/i);
return {
title: titleMatch ? titleMatch[1].trim() : 'Unknown Title',
description: descriptionMatch ? descriptionMatch[1].trim() : 'No description available',
url,
source: new URL(url).hostname,
extractedAt: new Date().toISOString(),
};
}
/**
* Cleans HTML content for LLM context
* @param html The HTML content to clean
* @returns Cleaned content
*/
private cleanContent(html: string): string {
// Extract the main content
// In a real implementation, you would use a proper HTML parser
// For this simulation, we'll use a simple approach with regex
// Try to find the main content container
let content = html;
// Try to extract article content
const articleMatch = html.match(/<article[^>]*>([\s\S]*?)<\/article>/i);
if (articleMatch && articleMatch[1]) {
content = articleMatch[1];
} else {
// Try to extract main content
const mainMatch = html.match(/<main[^>]*>([\s\S]*?)<\/main>/i);
if (mainMatch && mainMatch[1]) {
content = mainMatch[1];
}
}
// Remove HTML tags but preserve headings and paragraph structure
content = content
// Replace headings with markdown-style headings
.replace(/<h1[^>]*>([\s\S]*?)<\/h1>/gi, '# $1\n\n')
.replace(/<h2[^>]*>([\s\S]*?)<\/h2>/gi, '## $1\n\n')
.replace(/<h3[^>]*>([\s\S]*?)<\/h3>/gi, '### $1\n\n')
.replace(/<h4[^>]*>([\s\S]*?)<\/h4>/gi, '#### $1\n\n')
.replace(/<h5[^>]*>([\s\S]*?)<\/h5>/gi, '##### $1\n\n')
.replace(/<h6[^>]*>([\s\S]*?)<\/h6>/gi, '###### $1\n\n')
// Replace list items with markdown-style list items
.replace(/<li[^>]*>([\s\S]*?)<\/li>/gi, '- $1\n')
// Replace paragraphs with newline-separated text
.replace(/<p[^>]*>([\s\S]*?)<\/p>/gi, '$1\n\n')
// Replace code blocks with markdown-style code blocks
.replace(/<pre[^>]*><code[^>]*>([\s\S]*?)<\/code><\/pre>/gi, '```\n$1\n```\n\n')
// Replace inline code with markdown-style inline code
.replace(/<code[^>]*>([\s\S]*?)<\/code>/gi, '`$1`')
// Replace links with markdown-style links
.replace(/<a[^>]*href="([^"]*)"[^>]*>([\s\S]*?)<\/a>/gi, '[$2]($1)')
// Replace strong/bold with markdown-style bold
.replace(/<(strong|b)[^>]*>([\s\S]*?)<\/(strong|b)>/gi, '**$2**')
// Replace emphasis/italic with markdown-style italic
.replace(/<(em|i)[^>]*>([\s\S]*?)<\/(em|i)>/gi, '*$2*')
// Remove all other HTML tags
.replace(/<[^>]*>/g, '')
// Fix multiple newlines
.replace(/\n{3,}/g, '\n\n')
// Decode HTML entities
.replace(/ /g, ' ')
.replace(/</g, '<')
.replace(/>/g, '>')
.replace(/&/g, '&')
.replace(/"/g, '"')
.replace(/'/g, "'")
// Trim whitespace
.trim();
return content;
}
/**
* Formats content for LLM context
* @param content The cleaned content
* @param metadata The metadata
* @returns Formatted content for LLM context
*/
private formatForLLM(content: string, metadata: Record<string, string>): string {
// Create a header with metadata
const header = `
Title: ${metadata.title}
Source: ${metadata.source}
URL: ${metadata.url}
Extracted: ${metadata.extractedAt}
Description: ${metadata.description}
---
`;
// Combine header and content
return header + content;
}
/**
* Summarizes content (in a real implementation, this would call an LLM API)
* @param content The content to summarize
* @param maxLength Maximum length of the summary
* @returns Summarized content
*/
summarizeContent(content: string, maxLength: number = 500): string {
// In a real implementation, you would call an LLM API here
console.log('Simulating content summarization...');
// For this simulation, we'll return a mock summary
const mockSummary = `
# Browser Rendering API Summary
Cloudflare Browser Rendering is a serverless headless browser service for Cloudflare Workers that enables:
1. Rendering JavaScript-heavy websites
2. Taking screenshots and generating PDFs
3. Extracting structured data
4. Automating browser interactions
It offers two main interfaces:
- **REST API**: Simple endpoints for common tasks
- **Workers Binding API**: Advanced integration with Puppeteer
The service runs within Cloudflare's network, providing low-latency access to browser capabilities without managing infrastructure.
`.trim();
// Truncate if necessary
return mockSummary.length > maxLength
? mockSummary.substring(0, maxLength) + '...'
: mockSummary;
}
}
```
--------------------------------------------------------------------------------
/puppeteer-worker.js:
--------------------------------------------------------------------------------
```javascript
import puppeteer from '@cloudflare/puppeteer';
/**
* Cloudflare Worker with Browser Rendering binding
*
* This worker demonstrates how to use the Browser Rendering binding
* with the @cloudflare/puppeteer package.
*/
// Define the allowed origins for CORS
const ALLOWED_ORIGINS = [
'https://example.com',
'http://localhost:3000',
];
/**
* Handle CORS preflight requests
*/
function handleOptions(request) {
const origin = request.headers.get('Origin');
const isAllowedOrigin = ALLOWED_ORIGINS.includes(origin);
return new Response(null, {
status: 204,
headers: {
'Access-Control-Allow-Origin': isAllowedOrigin ? origin : ALLOWED_ORIGINS[0],
'Access-Control-Allow-Methods': 'GET, POST, OPTIONS',
'Access-Control-Allow-Headers': 'Content-Type, Authorization',
'Access-Control-Max-Age': '86400',
},
});
}
/**
* Add CORS headers to a response
*/
function addCorsHeaders(response, request) {
const origin = request.headers.get('Origin');
const isAllowedOrigin = ALLOWED_ORIGINS.includes(origin);
const headers = new Headers(response.headers);
headers.set('Access-Control-Allow-Origin', isAllowedOrigin ? origin : ALLOWED_ORIGINS[0]);
return new Response(response.body, {
status: response.status,
statusText: response.statusText,
headers,
});
}
/**
* Handle the /content endpoint
*/
async function handleContent(request, env) {
try {
// Parse the request body
const body = await request.json();
if (!body.url) {
return new Response(JSON.stringify({ error: 'URL is required' }), {
status: 400,
headers: { 'Content-Type': 'application/json' },
});
}
// Launch a browser using the binding
const browser = await puppeteer.launch(env.browser);
try {
// Create a new page
const page = await browser.newPage();
// Set viewport size
await page.setViewport({
width: 1280,
height: 800,
});
// Set request rejection patterns if provided
if (body.rejectResourceTypes && Array.isArray(body.rejectResourceTypes)) {
await page.setRequestInterception(true);
page.on('request', (req) => {
if (body.rejectResourceTypes.includes(req.resourceType())) {
req.abort();
} else {
req.continue();
}
});
}
// Navigate to the URL
await page.goto(body.url, {
waitUntil: body.waitUntil || 'networkidle0',
timeout: body.timeout || 30000,
});
// Get the page content
const content = await page.content();
// Return the content
return new Response(JSON.stringify({ content }), {
headers: { 'Content-Type': 'application/json' },
});
} finally {
// Always close the browser to avoid resource leaks
await browser.close();
}
} catch (error) {
return new Response(JSON.stringify({ error: error.message, stack: error.stack }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
}
/**
* Handle the /screenshot endpoint
*/
async function handleScreenshot(request, env) {
try {
// Parse the request body
const body = await request.json();
if (!body.url) {
return new Response(JSON.stringify({ error: 'URL is required' }), {
status: 400,
headers: { 'Content-Type': 'application/json' },
});
}
// Launch a browser using the binding
const browser = await puppeteer.launch(env.browser);
try {
// Create a new page
const page = await browser.newPage();
// Set viewport size
await page.setViewport({
width: body.width || 1280,
height: body.height || 800,
});
// Navigate to the URL
await page.goto(body.url, {
waitUntil: body.waitUntil || 'networkidle0',
timeout: body.timeout || 30000,
});
// Take a screenshot
const screenshot = await page.screenshot({
fullPage: body.fullPage || false,
type: 'png',
encoding: 'base64',
});
// Return the screenshot
return new Response(JSON.stringify({ screenshot }), {
headers: { 'Content-Type': 'application/json' },
});
} finally {
// Always close the browser to avoid resource leaks
await browser.close();
}
} catch (error) {
return new Response(JSON.stringify({ error: error.message, stack: error.stack }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
}
/**
* Main worker handler
*/
export default {
async fetch(request, env, ctx) {
// Handle CORS preflight requests
if (request.method === 'OPTIONS') {
return handleOptions(request);
}
// Get the URL pathname
const url = new URL(request.url);
const path = url.pathname.toLowerCase();
// Route the request to the appropriate handler
let response;
try {
if (path === '/content' && request.method === 'POST') {
response = await handleContent(request, env);
} else if (path === '/screenshot' && request.method === 'POST') {
response = await handleScreenshot(request, env);
} else if (path === '/info') {
// Return information about the environment
response = new Response(JSON.stringify({
env: Object.keys(env),
hasBrowser: 'browser' in env,
puppeteer: {
version: puppeteer.version,
},
}, null, 2), {
headers: { 'Content-Type': 'application/json' },
});
} else {
response = new Response(JSON.stringify({
error: 'Endpoint not found',
availableEndpoints: ['/content', '/screenshot', '/info'],
method: request.method,
}), {
status: 404,
headers: { 'Content-Type': 'application/json' },
});
}
} catch (error) {
response = new Response(JSON.stringify({
error: error.message,
stack: error.stack,
}), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
// Add CORS headers to the response
return addCorsHeaders(response, request);
},
};
```
--------------------------------------------------------------------------------
/experiments/puppeteer-binding/index.ts:
--------------------------------------------------------------------------------
```typescript
/**
* Experiment: Cloudflare Browser Rendering Workers Binding API with Puppeteer
*
* Note: This is a simulation of how you would use the Cloudflare Browser Rendering
* Workers Binding API with Puppeteer. In a real implementation, this code would
* run within a Cloudflare Worker with the Browser Rendering binding.
*/
// In a real Cloudflare Worker, you would import Puppeteer like this:
// import puppeteer from '@cloudflare/puppeteer';
/**
* Simulated function to navigate through Cloudflare documentation and extract structured information
*/
async function navigateAndExtractContent() {
console.log('Simulating Puppeteer navigation and content extraction...');
// In a real implementation, you would initialize Puppeteer like this:
/*
const browser = await puppeteer.launch({
// Browser Rendering specific options
userDataDir: '/tmp/puppeteer_user_data',
});
try {
const page = await browser.newPage();
// Navigate to Cloudflare docs
await page.goto('https://developers.cloudflare.com/browser-rendering/', {
waitUntil: 'networkidle0',
});
// Extract headings
const headings = await page.evaluate(() => {
const headingElements = document.querySelectorAll('h1, h2, h3');
return Array.from(headingElements).map(el => ({
level: el.tagName.toLowerCase(),
text: el.textContent?.trim() || '',
}));
});
// Extract code examples
const codeExamples = await page.evaluate(() => {
const codeElements = document.querySelectorAll('pre code');
return Array.from(codeElements).map(el => ({
language: el.className.replace('language-', ''),
code: el.textContent?.trim() || '',
}));
});
// Navigate to a different section
await page.click('a[href*="rest-api"]');
await page.waitForNavigation({ waitUntil: 'networkidle0' });
// Extract API endpoints
const apiEndpoints = await page.evaluate(() => {
const endpointElements = document.querySelectorAll('.endpoint');
return Array.from(endpointElements).map(el => ({
method: el.querySelector('.method')?.textContent?.trim() || '',
path: el.querySelector('.path')?.textContent?.trim() || '',
description: el.querySelector('.description')?.textContent?.trim() || '',
}));
});
return {
headings,
codeExamples,
apiEndpoints,
};
} finally {
// In a real implementation with session reuse, you would use:
// await browser.disconnect();
// Instead of:
// await browser.close();
}
*/
// For this simulation, we'll return mock data
return {
headings: [
{ level: 'h1', text: 'Browser Rendering' },
{ level: 'h2', text: 'Overview' },
{ level: 'h2', text: 'REST API' },
{ level: 'h3', text: 'Content Endpoint' },
{ level: 'h3', text: 'Screenshot Endpoint' },
{ level: 'h2', text: 'Workers Binding API' },
],
codeExamples: [
{
language: 'javascript',
code: `
// Example of using the REST API
fetch('https://browser-rendering.example.workers.dev/content', {
method: 'POST',
body: JSON.stringify({
url: 'https://example.com',
rejectResourceTypes: ['image', 'font']
})
})
.then(response => response.json())
.then(data => console.log(data.content));
`
},
{
language: 'javascript',
code: `
// Example of using the Workers Binding API
import puppeteer from '@cloudflare/puppeteer';
export default {
async fetch(request, env) {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
const content = await page.content();
await browser.disconnect();
return new Response(content);
}
};
`
}
],
apiEndpoints: [
{
method: 'POST',
path: '/content',
description: 'Fetches rendered HTML content from a URL'
},
{
method: 'POST',
path: '/screenshot',
description: 'Captures a screenshot of a web page'
},
{
method: 'POST',
path: '/pdf',
description: 'Renders a web page as a PDF document'
},
{
method: 'POST',
path: '/scrape',
description: 'Extracts structured data from HTML elements'
}
]
};
}
/**
* Simulated function to demonstrate session reuse
*/
async function demonstrateSessionReuse() {
console.log('Simulating Puppeteer session reuse...');
// In a real implementation, you would use code like this:
/*
// Get existing browser sessions
const sessions = await puppeteer.sessions();
let browser;
if (sessions.length > 0) {
// Connect to an existing session
browser = await puppeteer.connect({ sessionId: sessions[0].id });
console.log('Connected to existing session');
} else {
// Create a new session
browser = await puppeteer.launch();
console.log('Created new session');
}
try {
// Use the browser...
const page = await browser.newPage();
await page.goto('https://example.com');
// ...
} finally {
// Disconnect instead of closing to keep the session alive
await browser.disconnect();
}
*/
console.log('In a real implementation, you would:');
console.log('1. Check for existing sessions with puppeteer.sessions()');
console.log('2. Connect to an existing session or create a new one');
console.log('3. Use browser.disconnect() instead of browser.close() to keep the session alive');
}
/**
* Main function to run the experiment
*/
async function runExperiment() {
console.log('Starting Browser Rendering Workers Binding API experiment...');
try {
// Simulate navigating and extracting content
const extractedData = await navigateAndExtractContent();
// Display the extracted data
console.log('\nExtracted headings:');
extractedData.headings.forEach(heading => {
console.log(`${heading.level}: ${heading.text}`);
});
console.log('\nExtracted API endpoints:');
extractedData.apiEndpoints.forEach(endpoint => {
console.log(`${endpoint.method} ${endpoint.path} - ${endpoint.description}`);
});
console.log('\nExtracted code examples (first example):');
if (extractedData.codeExamples.length > 0) {
console.log(`Language: ${extractedData.codeExamples[0].language}`);
console.log(extractedData.codeExamples[0].code);
}
// Simulate session reuse
await demonstrateSessionReuse();
console.log('\nNote: This is a simulation. In a real implementation:');
console.log('1. This code would run within a Cloudflare Worker');
console.log('2. You would use the actual @cloudflare/puppeteer package');
console.log('3. You would need to set up the Browser Rendering binding in your wrangler.toml');
} catch (error) {
console.error('Experiment failed:', error);
}
}
// Run the experiment
runExperiment();
```
--------------------------------------------------------------------------------
/experiments/content-extraction/index.ts:
--------------------------------------------------------------------------------
```typescript
import axios from 'axios';
/**
* Experiment: Content Extraction and Processing for LLM Context
*
* This experiment demonstrates how to extract and process web content
* specifically for use as context in LLMs using Cloudflare Browser Rendering.
*/
/**
* Simulated function to extract clean content from a web page
* @param url The URL to extract content from
*/
async function extractCleanContent(url: string) {
console.log(`Simulating content extraction from: ${url}`);
// In a real implementation with Cloudflare Browser Rendering, you would:
// 1. Use the /content endpoint to get the rendered HTML
// 2. Use the /scrape endpoint to extract specific elements
// 3. Process the content to make it suitable for LLM context
// Simulated HTML content from Cloudflare docs
const simulatedHtml = `
<!DOCTYPE html>
<html>
<head>
<title>Browser Rendering API | Cloudflare Docs</title>
<meta name="description" content="Learn how to use Cloudflare's Browser Rendering API">
</head>
<body>
<header>
<nav>
<ul>
<li><a href="/">Home</a></li>
<li><a href="/products">Products</a></li>
<li><a href="/developers">Developers</a></li>
</ul>
</nav>
</header>
<main>
<article>
<h1>Browser Rendering API</h1>
<p class="description">Cloudflare Browser Rendering is a serverless headless browser service that allows execution of browser actions within Cloudflare Workers.</p>
<section id="overview">
<h2>Overview</h2>
<p>Browser Rendering allows you to run a headless browser directly within Cloudflare's network, enabling you to:</p>
<ul>
<li>Render JavaScript-heavy websites</li>
<li>Take screenshots of web pages</li>
<li>Generate PDFs</li>
<li>Extract structured data</li>
<li>Automate browser interactions</li>
</ul>
</section>
<section id="rest-api">
<h2>REST API</h2>
<p>The REST API provides simple endpoints for common browser tasks:</p>
<div class="endpoint">
<h3>/content</h3>
<p>Fetches rendered HTML content from a URL after JavaScript execution.</p>
<pre><code>
POST /content
{
"url": "https://example.com",
"rejectResourceTypes": ["image", "font"]
}
</code></pre>
</div>
<div class="endpoint">
<h3>/screenshot</h3>
<p>Captures a screenshot of a web page.</p>
</div>
</section>
<section id="workers-binding">
<h2>Workers Binding API</h2>
<p>For more advanced use cases, you can use the Workers Binding API with Puppeteer.</p>
<pre><code>
import puppeteer from '@cloudflare/puppeteer';
export default {
async fetch(request, env) {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
const content = await page.content();
await browser.disconnect();
return new Response(content);
}
};
</code></pre>
</section>
</article>
</main>
<footer>
<p>© 2025 Cloudflare, Inc.</p>
<nav>
<ul>
<li><a href="/terms">Terms</a></li>
<li><a href="/privacy">Privacy</a></li>
</ul>
</nav>
</footer>
</body>
</html>
`;
return simulatedHtml;
}
/**
* Extracts main content from HTML and removes unnecessary elements
* @param html The HTML content
* @returns Cleaned content suitable for LLM context
*/
function cleanContentForLLM(html: string): string {
// In a real implementation, you would use a proper HTML parser
// For this experiment, we'll use a simple approach with regex
// Extract the article content
const articleMatch = html.match(/<article[^>]*>([\s\S]*?)<\/article>/i);
let content = articleMatch ? articleMatch[1] : html;
// Remove HTML tags but preserve headings and paragraph structure
content = content
// Replace headings with markdown-style headings
.replace(/<h1[^>]*>([\s\S]*?)<\/h1>/gi, '# $1\n\n')
.replace(/<h2[^>]*>([\s\S]*?)<\/h2>/gi, '## $1\n\n')
.replace(/<h3[^>]*>([\s\S]*?)<\/h3>/gi, '### $1\n\n')
// Replace list items with markdown-style list items
.replace(/<li[^>]*>([\s\S]*?)<\/li>/gi, '- $1\n')
// Replace paragraphs with newline-separated text
.replace(/<p[^>]*>([\s\S]*?)<\/p>/gi, '$1\n\n')
// Replace code blocks with markdown-style code blocks
.replace(/<pre[^>]*><code[^>]*>([\s\S]*?)<\/code><\/pre>/gi, '```\n$1\n```\n\n')
// Remove all other HTML tags
.replace(/<[^>]*>/g, '')
// Fix multiple newlines
.replace(/\n{3,}/g, '\n\n')
// Trim whitespace
.trim();
return content;
}
/**
* Extracts metadata from HTML
* @param html The HTML content
* @returns Metadata object
*/
function extractMetadata(html: string): Record<string, string> {
const titleMatch = html.match(/<title[^>]*>([\s\S]*?)<\/title>/i);
const descriptionMatch = html.match(/<meta name="description" content="([^"]*)">/i);
return {
title: titleMatch ? titleMatch[1].trim() : 'Unknown Title',
description: descriptionMatch ? descriptionMatch[1].trim() : 'No description available',
url: 'https://developers.cloudflare.com/browser-rendering/', // Simulated URL
source: 'Cloudflare Documentation',
extractedAt: new Date().toISOString(),
};
}
/**
* Formats content for LLM context
* @param content The cleaned content
* @param metadata The metadata
* @returns Formatted content for LLM context
*/
function formatForLLMContext(content: string, metadata: Record<string, string>): string {
// Create a header with metadata
const header = `
Title: ${metadata.title}
Source: ${metadata.source}
URL: ${metadata.url}
Extracted: ${metadata.extractedAt}
Description: ${metadata.description}
---
`;
// Combine header and content
return header + content;
}
/**
* Simulates content summarization using an LLM
* @param content The content to summarize
* @returns Summarized content
*/
function simulateLLMSummarization(content: string): string {
// In a real implementation, you would call an LLM API here
console.log('Simulating LLM summarization...');
// For this simulation, we'll return a mock summary
return `
# Browser Rendering API Summary
Cloudflare Browser Rendering is a serverless headless browser service for Cloudflare Workers that enables:
1. Rendering JavaScript-heavy websites
2. Taking screenshots and generating PDFs
3. Extracting structured data
4. Automating browser interactions
It offers two main interfaces:
- **REST API**: Simple endpoints for common tasks like fetching content (/content) and taking screenshots (/screenshot)
- **Workers Binding API**: Advanced integration with Puppeteer for complex automation
The service runs within Cloudflare's network, providing low-latency access to browser capabilities without managing infrastructure.
`;
}
/**
* Main function to run the experiment
*/
async function runExperiment() {
console.log('Starting Content Extraction and Processing experiment...');
try {
// Extract content from Cloudflare docs
const url = 'https://developers.cloudflare.com/browser-rendering/';
const html = await extractCleanContent(url);
// Clean the content for LLM context
const cleanedContent = cleanContentForLLM(html);
console.log('\nCleaned content for LLM:');
console.log(cleanedContent.substring(0, 500) + '...');
// Extract metadata
const metadata = extractMetadata(html);
console.log('\nExtracted metadata:');
console.log(metadata);
// Format for LLM context
const formattedContent = formatForLLMContext(cleanedContent, metadata);
console.log('\nFormatted content for LLM context:');
console.log(formattedContent.substring(0, 300) + '...');
// Simulate LLM summarization
const summarizedContent = simulateLLMSummarization(formattedContent);
console.log('\nSimulated LLM summarization:');
console.log(summarizedContent);
console.log('\nIn a real implementation, you would:');
console.log('1. Use Cloudflare Browser Rendering to fetch the actual content');
console.log('2. Use a proper HTML parser for content extraction');
console.log('3. Call a real LLM API for summarization');
console.log('4. Store the processed content in Cloudflare R2 or another storage solution');
} catch (error) {
console.error('Experiment failed:', error);
}
}
// Run the experiment
runExperiment();
```
--------------------------------------------------------------------------------
/src/server.ts:
--------------------------------------------------------------------------------
```typescript
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
import {
CallToolRequestSchema,
ErrorCode,
ListToolsRequestSchema,
McpError,
} from '@modelcontextprotocol/sdk/types.js';
import { BrowserClient } from './browser-client.js';
import { ContentProcessor } from './content-processor.js';
/**
* Cloudflare Browser Rendering MCP Server
*
* This server provides tools for fetching and processing web content
* using Cloudflare Browser Rendering for use as context in LLMs.
*/
export class BrowserRenderingServer {
private server: Server;
private browserClient: BrowserClient;
private contentProcessor: ContentProcessor;
constructor() {
this.server = new Server(
{
name: 'cloudflare-browser-rendering',
version: '0.1.0',
},
{
capabilities: {
tools: {},
},
}
);
// Initialize the browser client and content processor
this.browserClient = new BrowserClient();
this.contentProcessor = new ContentProcessor();
// Set up request handlers
this.setupToolHandlers();
// Error handling
this.server.onerror = (error) => console.error('[MCP Error]', error);
process.on('SIGINT', async () => {
await this.server.close();
process.exit(0);
});
}
/**
* Set up tool handlers for the MCP server
*/
private setupToolHandlers() {
// List available tools
this.server.setRequestHandler(ListToolsRequestSchema, async () => ({
tools: [
{
name: 'fetch_page',
description: 'Fetches and processes a web page for LLM context',
inputSchema: {
type: 'object',
properties: {
url: {
type: 'string',
description: 'URL to fetch',
},
includeScreenshot: {
type: 'boolean',
description: 'Whether to include a screenshot (base64 encoded)',
},
maxContentLength: {
type: 'number',
description: 'Maximum content length to return',
},
},
required: ['url'],
},
},
{
name: 'search_documentation',
description: 'Searches Cloudflare documentation and returns relevant content',
inputSchema: {
type: 'object',
properties: {
query: {
type: 'string',
description: 'Search query',
},
maxResults: {
type: 'number',
description: 'Maximum number of results to return',
},
},
required: ['query'],
},
},
{
name: 'extract_structured_content',
description: 'Extracts structured content from a web page using CSS selectors',
inputSchema: {
type: 'object',
properties: {
url: {
type: 'string',
description: 'URL to extract content from',
},
selectors: {
type: 'object',
description: 'CSS selectors to extract content',
additionalProperties: {
type: 'string',
},
},
},
required: ['url', 'selectors'],
},
},
{
name: 'summarize_content',
description: 'Summarizes web content for more concise LLM context',
inputSchema: {
type: 'object',
properties: {
url: {
type: 'string',
description: 'URL to summarize',
},
maxLength: {
type: 'number',
description: 'Maximum length of the summary',
},
},
required: ['url'],
},
},
],
}));
// Handle tool calls
this.server.setRequestHandler(CallToolRequestSchema, async (request) => {
const { name, arguments: args } = request.params;
try {
switch (name) {
case 'fetch_page':
return await this.handleFetchPage(args);
case 'search_documentation':
return await this.handleSearchDocumentation(args);
case 'extract_structured_content':
return await this.handleExtractStructuredContent(args);
case 'summarize_content':
return await this.handleSummarizeContent(args);
default:
throw new McpError(
ErrorCode.MethodNotFound,
`Unknown tool: ${name}`
);
}
} catch (error) {
if (error instanceof McpError) {
throw error;
}
console.error(`Error in tool ${name}:`, error);
throw new McpError(
ErrorCode.InternalError,
`Error executing tool ${name}: ${error instanceof Error ? error.message : String(error)}`
);
}
});
}
/**
* Handle the fetch_page tool
*/
private async handleFetchPage(args: any) {
// Validate arguments
if (typeof args !== 'object' || args === null || typeof args.url !== 'string') {
throw new McpError(ErrorCode.InvalidParams, 'Invalid arguments for fetch_page');
}
const { url, includeScreenshot = false, maxContentLength = 10000 } = args;
try {
// Fetch the page content
const html = await this.browserClient.fetchContent(url);
// Process the content for LLM
const processedContent = this.contentProcessor.processForLLM(html, url);
// Truncate if necessary
const truncatedContent = processedContent.length > maxContentLength
? processedContent.substring(0, maxContentLength) + '...'
: processedContent;
// Get screenshot if requested
let screenshot = null;
if (includeScreenshot) {
screenshot = await this.browserClient.takeScreenshot(url);
}
// Return the result
return {
content: [
{
type: 'text',
text: truncatedContent,
},
...(screenshot ? [{
type: 'image',
image: screenshot,
}] : []),
],
};
} catch (error) {
console.error('Error fetching page:', error);
return {
content: [
{
type: 'text',
text: `Error fetching page: ${error instanceof Error ? error.message : String(error)}`,
},
],
isError: true,
};
}
}
/**
* Handle the search_documentation tool
*/
private async handleSearchDocumentation(args: any) {
// Validate arguments
if (typeof args !== 'object' || args === null || typeof args.query !== 'string') {
throw new McpError(ErrorCode.InvalidParams, 'Invalid arguments for search_documentation');
}
const { query, maxResults = 3 } = args;
try {
// In a real implementation, you would:
// 1. Use Cloudflare Browser Rendering to navigate to the docs
// 2. Use the search functionality on the docs site
// 3. Extract the search results
// For this simulation, we'll return mock results
const mockResults = [
{
title: 'Browser Rendering API Overview',
url: 'https://developers.cloudflare.com/browser-rendering/',
snippet: 'Cloudflare Browser Rendering is a serverless headless browser service that allows execution of browser actions within Cloudflare Workers.',
},
{
title: 'REST API Reference',
url: 'https://developers.cloudflare.com/browser-rendering/rest-api/',
snippet: 'The REST API provides simple endpoints for common browser tasks like fetching content, taking screenshots, and generating PDFs.',
},
{
title: 'Workers Binding API Reference',
url: 'https://developers.cloudflare.com/browser-rendering/workers-binding/',
snippet: 'For more advanced use cases, you can use the Workers Binding API with Puppeteer to automate browser interactions.',
},
].slice(0, maxResults);
// Format the results
const formattedResults = mockResults.map(result =>
`## [${result.title}](${result.url})\n${result.snippet}\n`
).join('\n');
return {
content: [
{
type: 'text',
text: `# Search Results for "${query}"\n\n${formattedResults}`,
},
],
};
} catch (error) {
console.error('Error searching documentation:', error);
return {
content: [
{
type: 'text',
text: `Error searching documentation: ${error instanceof Error ? error.message : String(error)}`,
},
],
isError: true,
};
}
}
/**
* Handle the extract_structured_content tool
*/
private async handleExtractStructuredContent(args: any) {
// Validate arguments
if (
typeof args !== 'object' ||
args === null ||
typeof args.url !== 'string' ||
typeof args.selectors !== 'object'
) {
throw new McpError(ErrorCode.InvalidParams, 'Invalid arguments for extract_structured_content');
}
const { url, selectors } = args;
try {
// In a real implementation, you would:
// 1. Use Cloudflare Browser Rendering to fetch the page
// 2. Use the /scrape endpoint to extract content based on selectors
// For this simulation, we'll return mock results
const mockResults: Record<string, string> = {};
for (const [key, selector] of Object.entries(selectors)) {
if (typeof selector === 'string') {
// Simulate extraction based on selector
mockResults[key] = `Extracted content for selector "${selector}"`;
}
}
// Format the results
const formattedResults = Object.entries(mockResults)
.map(([key, value]) => `## ${key}\n${value}`)
.join('\n\n');
return {
content: [
{
type: 'text',
text: `# Structured Content from ${url}\n\n${formattedResults}`,
},
],
};
} catch (error) {
console.error('Error extracting structured content:', error);
return {
content: [
{
type: 'text',
text: `Error extracting structured content: ${error instanceof Error ? error.message : String(error)}`,
},
],
isError: true,
};
}
}
/**
* Handle the summarize_content tool
*/
private async handleSummarizeContent(args: any) {
// Validate arguments
if (typeof args !== 'object' || args === null || typeof args.url !== 'string') {
throw new McpError(ErrorCode.InvalidParams, 'Invalid arguments for summarize_content');
}
const { url, maxLength = 500 } = args;
try {
// In a real implementation, you would:
// 1. Fetch the page content using Cloudflare Browser Rendering
// 2. Process the content for LLM
// 3. Call an LLM API to summarize the content
// For this simulation, we'll return a mock summary
const mockSummary = `
# Browser Rendering API Summary
Cloudflare Browser Rendering is a serverless headless browser service for Cloudflare Workers that enables:
1. Rendering JavaScript-heavy websites
2. Taking screenshots and generating PDFs
3. Extracting structured data
4. Automating browser interactions
It offers two main interfaces:
- **REST API**: Simple endpoints for common tasks
- **Workers Binding API**: Advanced integration with Puppeteer
The service runs within Cloudflare's network, providing low-latency access to browser capabilities without managing infrastructure.
`.trim();
// Truncate if necessary
const truncatedSummary = mockSummary.length > maxLength
? mockSummary.substring(0, maxLength) + '...'
: mockSummary;
return {
content: [
{
type: 'text',
text: truncatedSummary,
},
],
};
} catch (error) {
console.error('Error summarizing content:', error);
return {
content: [
{
type: 'text',
text: `Error summarizing content: ${error instanceof Error ? error.message : String(error)}`,
},
],
isError: true,
};
}
}
/**
* Run the MCP server
*/
async run() {
const transport = new StdioServerTransport();
await this.server.connect(transport);
}
}
```