# Directory Structure

```
├── changelog.md
├── LICENSE
├── pyproject.toml
├── README.md
└── src
    └── mcp_web_browser
        └── server.py
```

# Files

--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------

```markdown
# MCP Web Browser Server

An advanced web browsing server for the Model Context Protocol (MCP) powered by Playwright, enabling headless browser interactions through a flexible, secure API.

<a href="https://glama.ai/mcp/servers/lwqlaw6k6d"><img width="380" height="200" src="https://glama.ai/mcp/servers/lwqlaw6k6d/badge" alt="Web Browser Server MCP server" /></a>

## 🌐 Features

- **Headless Web Browsing**: Navigate to any website with SSL certificate validation bypass
- **Full Page Content Extraction**: Retrieve complete HTML content, including dynamically loaded JavaScript
- **Multi-Tab Support**: Create, manage, and switch between multiple browser tabs
- **Advanced Web Interaction Tools**:
  - Extract text content
  - Click page elements
  - Input text into form fields
  - Capture screenshots
  - Extract page links with filtering capabilities
  - Scroll pages in any direction
  - Execute JavaScript on pages
  - Refresh pages
  - Wait for navigation to complete
- **Resource Management**: Automatic cleanup of unused resources after inactivity
- **Enhanced Page Information**: Get detailed metadata about the current page

## 🚀 Quick Start

### Prerequisites

- Python 3.10+
- MCP SDK
- Playwright

### Installation

```bash
# Install MCP and Playwright
pip install mcp playwright

# Install browser dependencies
playwright install
```

### Configuration for Claude Desktop

Add to your `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "web-browser": {
      "command": "python",
      "args": [
        "/path/to/your/server.py"
      ]
    }
  }
}
```

## 💡 Usage Examples

### Basic Web Navigation

```python
# Browse to a website
page_content = browse_to("https://example.com")

# Extract page text
text_content = extract_text_content()

# Extract text from a specific element
title_text = extract_text_content("h1.title")
```

### Web Interaction

```python
# Navigate to a page
browse_to("https://example.com/login")

# Input text into a form
input_text("#username", "your_username")
input_text("#password", "your_password")

# Click a login button
click_element("#login-button")
```

### Screenshot Capture

```python
# Capture full page screenshot
full_page_screenshot = get_page_screenshots(full_page=True)

# Capture specific element screenshot
element_screenshot = get_page_screenshots(selector="#main-content")
```

### Link Extraction

```python
# Get all links on the page
page_links = get_page_links()

# Get links matching a pattern
filtered_links = get_page_links(filter_pattern="contact")
```

### Multi-Tab Browsing

```python
# Create a new tab
tab_id = create_new_tab("https://example.com")

# Create another tab
another_tab_id = create_new_tab("https://example.org")

# List all open tabs
tabs = list_tabs()

# Switch between tabs
switch_tab(tab_id)

# Close a tab
close_tab(another_tab_id)
```

### Advanced Interactions

```python
# Scroll the page
scroll_page(direction="down", amount="page")

# Execute JavaScript on the page
result = execute_javascript("return document.title")

# Get detailed page information
page_info = get_page_info()

# Refresh the current page
refresh_page()

# Wait for navigation to complete
wait_for_navigation(timeout_ms=5000)
```

## 🛡️ Security Features

- SSL certificate validation bypass
- Secure browser context management
- Custom user-agent configuration
- Error handling and comprehensive logging
- Configurable timeout settings
- CSP bypass control
- Protection against cookie stealing

## 🔧 Troubleshooting

### Common Issues

- **SSL Certificate Errors**: Automatically bypassed
- **Slow Page Load**: Adjust timeout in `browse_to()` method
- **Element Not Found**: Verify selectors carefully
- **Browser Resource Usage**: Auto-cleanup after inactivity period

### Logging

All significant events are logged with detailed information for easy debugging.

## 📋 Tool Parameters

### `browse_to(url: str, context: Optional[Any] = None)`
- `url`: Website to navigate to
- `context`: Optional context object (currently unused)

### `extract_text_content(selector: Optional[str] = None, context: Optional[Any] = None)`
- `selector`: Optional CSS selector to extract specific content
- `context`: Optional context object (currently unused)

### `click_element(selector: str, context: Optional[Any] = None)`
- `selector`: CSS selector of the element to click
- `context`: Optional context object (currently unused)

### `get_page_screenshots(full_page: bool = False, selector: Optional[str] = None, context: Optional[Any] = None)`
- `full_page`: Capture entire page screenshot
- `selector`: Optional element to screenshot
- `context`: Optional context object (currently unused)

### `get_page_links(filter_pattern: Optional[str] = None, context: Optional[Any] = None)`
- `filter_pattern`: Optional text pattern to filter links
- `context`: Optional context object (currently unused)

### `input_text(selector: str, text: str, context: Optional[Any] = None)`
- `selector`: CSS selector of input element
- `text`: Text to input
- `context`: Optional context object (currently unused)

### `create_new_tab(url: Optional[str] = None, context: Optional[Any] = None)`
- `url`: Optional URL to navigate to in the new tab
- `context`: Optional context object (currently unused)

### `switch_tab(tab_id: str, context: Optional[Any] = None)`
- `tab_id`: ID of the tab to switch to
- `context`: Optional context object (currently unused)

### `list_tabs(context: Optional[Any] = None)`
- `context`: Optional context object (currently unused)

### `close_tab(tab_id: Optional[str] = None, context: Optional[Any] = None)`
- `tab_id`: Optional ID of the tab to close (defaults to current tab)
- `context`: Optional context object (currently unused)

### `refresh_page(context: Optional[Any] = None)`
- `context`: Optional context object (currently unused)

### `get_page_info(context: Optional[Any] = None)`
- `context`: Optional context object (currently unused)

### `scroll_page(direction: str = "down", amount: str = "page", context: Optional[Any] = None)`
- `direction`: Direction to scroll ('up', 'down', 'left', 'right')
- `amount`: Amount to scroll ('page', 'half', or a number)
- `context`: Optional context object (currently unused)

### `wait_for_navigation(timeout_ms: int = 10000, context: Optional[Any] = None)`
- `timeout_ms`: Maximum time to wait in milliseconds
- `context`: Optional context object (currently unused)

### `execute_javascript(script: str, context: Optional[Any] = None)`
- `script`: JavaScript code to execute
- `context`: Optional context object (currently unused)

## 🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

### Development Setup

```bash
# Clone the repository
git clone https://github.com/random-robbie/mcp-web-browser.git

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows use `venv\Scripts\activate`

# Install dependencies
pip install -e .[dev]
```

## 📄 License

MIT License

## 🔗 Related Projects

- [Model Context Protocol](https://modelcontextprotocol.io)
- [Playwright](https://playwright.dev)
- [Claude Desktop](https://claude.ai/desktop)

## 💬 Support

For issues and questions, please [open an issue](https://github.com/random-robbie/mcp-web-browser/issues) on GitHub.

```

--------------------------------------------------------------------------------
/pyproject.toml:
--------------------------------------------------------------------------------

```toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "mcp-web-browser"
version = "0.1.0"
description = "MCP Web Browser Server using Playwright"
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
    "mcp[cli]",
    "playwright",
]

[project.optional-dependencies]
dev = [
    "pytest",
    "ruff",
]

[tool.hatch.build.targets.wheel]
packages = ["src/mcp_web_browser"]

[tool.hatch.envs.default]
dependencies = [
    "pytest",
    "ruff",
]

[tool.ruff]
line-length = 100
target-version = "py310"

[tool.ruff.lint]
select = ["E", "F", "W", "I", "N", "UP", "ASYNC"]
ignore = ["E501"]

[project.scripts]
mcp-web-browser = "mcp_web_browser.server:main"

```

--------------------------------------------------------------------------------
/changelog.md:
--------------------------------------------------------------------------------

```markdown
Security Enhancements

Proper Error Handling and Logging: Added comprehensive logging with formatted output and proper error tracing
URL Sanitization: Automatically adds HTTPS prefix to URLs when needed
Content Security Policy Bypass Control: Added option to bypass CSP for better testing capabilities
Custom User-Agent: Configurable user-agent string
Security Headers: Added security-related HTTP headers to requests
JavaScript Execution Restrictions: Added validation to prevent cookie stealing via JavaScript

Performance Improvements

Browser Inactivity Timeout: Automatically closes browser after inactivity to free resources
Optimized Page Loading: Changed from 'networkidle' to 'domcontentloaded' for faster page loading
Resource Cleanup: Improved cleanup processes to ensure all resources are properly released
Retry Logic: Added retry mechanisms for page creation and element clicking
Efficient Screenshot Capture: Added JPEG compression for faster screenshot processing

Functional Additions

Multi-Tab Support: Added ability to create, switch between, and manage multiple tabs

create_new_tab: Opens a new browser tab
switch_tab: Switches between open tabs
list_tabs: Shows all open tabs and their information
close_tab: Closes a specific tab


Advanced Page Interaction:

refresh_page: Refreshes the current page
get_page_info: Provides detailed page information including metadata
scroll_page: Controls page scrolling in all directions
wait_for_navigation: Waits for page navigation to complete
execute_javascript: Safely executes JavaScript on the page


Enhanced Link Extraction:

Added filtering capabilities for links
Returns more metadata about links (title, text, etc.)


Improved Element Interaction:

Better visibility checking for elements
Automatic scrolling to ensure elements are in view
More reliable clicking with retry logic



Code Structure and Reliability

Better State Management: Added global state tracking for browser, pages, and tabs
Background Monitoring: Added inactivity monitor to clean up unused resources
Detailed Logging: Comprehensive logging throughout the system
Dynamic Playwright Import: Better error handling for Playwright import
Configurability: Added more configuration options for timeouts, viewport size, etc.

These improvements make the web browser MCP server more robust, secure, and feature-rich for your pentesting work, while ensuring resources are properly managed.

```

--------------------------------------------------------------------------------
/src/mcp_web_browser/server.py:
--------------------------------------------------------------------------------

```python
from typing import Optional, Union, Any
import asyncio
import base64
import sys
from mcp.server import FastMCP

# Create an MCP server for web browsing
mcp = FastMCP("Web Browser")

# Global browser and page management
_browser = None
_browser_context = None
_current_page = None
_playwright_instance = None

# Dynamic import of Playwright to avoid early import errors
def _import_playwright():
    from playwright.async_api import async_playwright
    return async_playwright

async def _ensure_browser():
    """Ensure a browser instance is available with SSL validation disabled"""
    global _browser, _browser_context, _playwright_instance
    
    if _browser is None:
        playwright_module = _import_playwright()
        _playwright_instance = await playwright_module().start()
        _browser = await _playwright_instance.chromium.launch()
        
        # Create a browser context that ignores HTTPS errors
        _browser_context = await _browser.new_context(
            ignore_https_errors=True,  # Ignore SSL certificate errors
        )
    return _browser, _browser_context

async def _close_current_page():
    """Close the current page if it exists"""
    global _current_page
    if _current_page:
        try:
            await _current_page.close()
        except Exception:
            pass
        _current_page = None

async def _safe_cleanup():
    """Safely clean up browser resources"""
    global _browser, _current_page, _browser_context, _playwright_instance
    
    try:
        if _current_page:
            try:
                await _current_page.close()
            except Exception:
                pass
        
        if _browser_context:
            try:
                await _browser_context.close()
            except Exception:
                pass
        
        if _browser:
            try:
                await _browser.close()
            except Exception:
                pass
        
        if _playwright_instance:
            try:
                await _playwright_instance.stop()
            except Exception:
                pass
    except Exception as e:
        # Log the error, but don't re-raise
        print(f"Error during cleanup: {e}", file=sys.stderr)
    finally:
        # Reset global variables
        _browser = None
        _browser_context = None
        _current_page = None
        _playwright_instance = None

@mcp.tool()
async def browse_to(url: str, context: Optional[Any] = None) -> str:
    """
    Navigate to a specific URL and return the page's HTML content.
    
    Args:
        url: The full URL to navigate to
        context: Optional context object for logging (ignored)
    
    Returns:
        The full HTML content of the page
    """
    global _current_page, _browser, _browser_context
    
    # Ensure browser is launched with SSL validation disabled
    _, browser_context = await _ensure_browser()
    
    # Close any existing page
    await _close_current_page()
    
    # Optional logging, but do nothing with context
    print(f"Navigating to {url}", file=sys.stderr)
    
    try:
        # Create a new page and navigate
        _current_page = await browser_context.new_page()
        
        # Additional options to handle various SSL/security issues
        await _current_page.goto(url, 
            wait_until='networkidle',
            timeout=30000,  # 30 seconds timeout
        )
        
        # Get full page content including dynamically loaded JavaScript
        page_content = await _current_page.content()
        
        # Optional: extract additional metadata
        try:
            title = await _current_page.title()
            print(f"Page title: {title}", file=sys.stderr)
        except Exception:
            pass
        
        return page_content
    
    except Exception as e:
        print(f"Error navigating to {url}: {e}", file=sys.stderr)
        raise

@mcp.tool()
async def extract_text_content(
    selector: Optional[str] = None, 
    context: Optional[Any] = None
) -> str:
    """
    Extract text content from the current page, optionally using a CSS selector.
    
    Args:
        selector: Optional CSS selector to target specific elements
        context: Optional context object for logging (ignored)
    
    Returns:
        Extracted text content
    """
    global _current_page
    
    if not _current_page:
        raise ValueError("No page is currently loaded. Use browse_to first.")
    
    try:
        if selector:
            # If selector is provided, extract text from matching elements
            elements = await _current_page.query_selector_all(selector)
            text_content = "\n".join([await el.inner_text() for el in elements])
            print(f"Extracted text from selector: {selector}", file=sys.stderr)
        else:
            # If no selector, extract all visible text from the page
            text_content = await _current_page.inner_text('body')
        
        return text_content
    
    except Exception as e:
        print(f"Error extracting text: {e}", file=sys.stderr)
        raise ValueError(f"Error extracting text: {e}")

@mcp.tool()
async def click_element(
    selector: str, 
    context: Optional[Any] = None
) -> str:
    """
    Click an element on the current page.
    
    Args:
        selector: CSS selector for the element to click
        context: Optional context object for logging (ignored)
    
    Returns:
        Confirmation message or error details
    """
    global _current_page
    
    if not _current_page:
        raise ValueError("No page is currently loaded. Use browse_to first.")
    
    try:
        element = await _current_page.query_selector(selector)
        if not element:
            raise ValueError(f"No element found with selector: {selector}")
        
        await element.click()
        print(f"Clicked element: {selector}", file=sys.stderr)
        
        return f"Successfully clicked element: {selector}"
    
    except Exception as e:
        print(f"Error clicking element: {e}", file=sys.stderr)
        raise ValueError(f"Error clicking element: {e}")

@mcp.tool()
async def get_page_screenshots(
    full_page: bool = False, 
    selector: Optional[str] = None,
    context: Optional[Any] = None
) -> str:
    """
    Capture screenshot of the current page.
    
    Args:
        full_page: Whether to capture the entire page or just the viewport
        selector: Optional CSS selector to screenshot a specific element
        context: Optional context object for logging (ignored)
    
    Returns:
        Base64 encoded screenshot image
    """
    global _current_page
    
    if not _current_page:
        raise ValueError("No page is currently loaded. Use browse_to first.")
    
    try:
        if selector:
            element = await _current_page.query_selector(selector)
            if not element:
                raise ValueError(f"No element found with selector: {selector}")
            screenshot_bytes = await element.screenshot()
        else:
            screenshot_bytes = await _current_page.screenshot(full_page=full_page)
        
        # Convert to base64 for easy transmission
        screenshot_base64 = base64.b64encode(screenshot_bytes).decode('utf-8')
        
        print(f"Screenshot captured: {'full page' if full_page else 'viewport'}", file=sys.stderr)
        
        return screenshot_base64
    
    except Exception as e:
        print(f"Error capturing screenshot: {e}", file=sys.stderr)
        raise ValueError(f"Error capturing screenshot: {e}")

@mcp.tool()
async def get_page_links(context: Optional[Any] = None) -> list[str]:
    """
    Extract all links from the current page.
    
    Args:
        context: Optional context object for logging (ignored)
    
    Returns:
        List of links found on the page
    """
    global _current_page
    
    if not _current_page:
        raise ValueError("No page is currently loaded. Use browse_to first.")
    
    try:
        # Use JavaScript to extract all links
        links = await _current_page.evaluate("""
            () => {
                const links = document.querySelectorAll('a');
                return Array.from(links).map(link => link.href);
            }
        """)
        
        print(f"Extracted {len(links)} links from the page", file=sys.stderr)
        
        return links
    
    except Exception as e:
        print(f"Error extracting links: {e}", file=sys.stderr)
        raise ValueError(f"Error extracting links: {e}")

@mcp.tool()
async def input_text(
    selector: str, 
    text: str, 
    context: Optional[Any] = None
) -> str:
    """
    Input text into a specific element on the page.
    
    Args:
        selector: CSS selector for the input element
        text: Text to input
        context: Optional context object for logging (ignored)
    
    Returns:
        Confirmation message
    """
    global _current_page
    
    if not _current_page:
        raise ValueError("No page is currently loaded. Use browse_to first.")
    
    try:
        element = await _current_page.query_selector(selector)
        if not element:
            raise ValueError(f"No element found with selector: {selector}")
        
        await element.fill(text)
        
        print(f"Input text into element: {selector}", file=sys.stderr)
        
        return f"Successfully input text into element: {selector}"
    
    except Exception as e:
        print(f"Error inputting text: {e}", file=sys.stderr)
        raise ValueError(f"Error inputting text: {e}")

def main():
    try:
        mcp.run()
    except Exception as e:
        print(f"Error running MCP server: {e}", file=sys.stderr)
    finally:
        # Use a separate event loop to ensure cleanup
        try:
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
            loop.run_until_complete(_safe_cleanup())
            loop.close()
        except Exception as cleanup_error:
            print(f"Cleanup error: {cleanup_error}", file=sys.stderr)

if __name__ == "__main__":
    main()

```